Anecdotal observation - experience matters with effective prompting

Background: Been working in the industry since ~2004. Solid two decades in all sorts of things orbiting data.

The past few months I've been leveraging VSCode/Claude Code to handle myriad data modeling, architecture, engineering, bi developer, documentation author, tasks. Some large projects, others one off. I think it's gone very well, quite successful with outcomes.

What's running through my head, as organizations roll these things out, is who is best suited to take real advantage of LLMs. My anecdotal observation is that experiences, seasoned people are more effective.

It makes sense - but I don't have a sense my own organization, and others, are looking at it like this. A company rolls out an LLM, it can go wide. Massive platform in the hands of people who don't yet have the capacity to understand the broader implications of their work. They don't have the experience of managing teams, structuring large chunks of work, prioritizing steps in order to achieve good results. All the things we pick up along the way that influence our ability to identify problems, work through and implement solutions.

edit: The other facet of this is ensuring a good pipeline of juniors, succession planning and the like. And I'm in this state of uncertainty on what is truly necessary for someone to learn and understand in order to do their work. And my comparison would be when I started off, the tools I had, being abstracted away from what people in the 70s or 80s would need to do. To write software, work with a database, it was much lower-level. We abstracted that away over time, and people didn't become incapable of understanding data, how it relates or drawing conclusions from it. The how changed.

So is it okay if someone is an effective prompter but doesn't understand the fundamentals of a star schema? I don't know.

There's no point here other than opening up for thoughts. I want to solidify something on this though, as it impacts some decisions I'm considering across the business. So your discussion here would be helpful.

Happy Friday!

It is just a tool. Like any other, you have to use it correctly for your needs, otherwise things will not meet your expectations. Garbage in, garbage out.

It is a force multiplier, it makes good people better.

mashie said:

It is a force multiplier, it makes good people better.

Click to expand...

Yep. Its making me so much more productive. Others at work seem to retarded to grasp it.

I've put LLMs in the hands of a couple people at my work. What caught me off guard were the novice users who would blindly follow it down really dumb paths, wasting untold time. It's an amazing tool but you need sufficient knowledge about you're working on to really reap the benefits.

equinox654 said:

Yep. Its making me so much more productive. Others at work seem to retarded to grasp it.

Click to expand...

retarded ???

For me, I would compare it to the "Search engine"

Many of us know people who claim "I Google'd that but couldn't find anything" , and then we go and Google it and first few results give us exactly what we were looking for.

LLM's are the same, sadly they have been sold as this magic solution to ALL your problems and too many people believe it.

For those of us who know how to use tools, analyze our usage, correct behavior to get better results, it is a god send!

One of my 2 recent uses that saved me a lot of time, was in 2 situations:
- Using Claude to help me architect a secured deployment for MCP hosting with our our existing Azure tenant to allow more integrations with some of our products while retaining control of usage and access, down to the very requirements for network isolation to keep it internal, products we can use and will use, and everything in between. I am of course vetting everything it said, but so far it has been 100% dead on.
- User account compromise - Fable and Opus , the amount of audit logs I have to pull and review, Claude made quick work of it, parsing, grouping, creating accurate timelines of events, confirming my audits I pulled from specific queries, including some items I missed in one audit KeyQL query I was using.

What often can take me several days (when other things come up of course), took me 2.5 days exactly and that was me being overly meticulous myself not wanting to miss anything!

philb2 said:

retarded ???

Click to expand...

"Please fix this excel file for me" - while not including what is wrong or what they want fixed, then getting upset when it doesn't know what to do, or just goes off and does something different.

This is due to the marketing and over hyping of the product, aswell as some people not wanting to learn how to prompt effectively.

We have some Teams channels for CoPilot and Claude, and I am always posting guides on usage, better prompting and how to try and get the most of the tools, but the questions some people still ask...

MrGuvernment said:

For me, I would compare it to the "Search engine"

Many of us know people who claim "I Google'd that but couldn't find anything" , and then we go and Google it and first few results give us exactly what we were looking for.

LLM's are the same, sadly they have been sold as this magic solution to ALL your problems and too many people believe it.

For those of us who know how to use tools, analyze our usage, correct behavior to get better results, it is a god send!

One of my 2 recent uses that saved me a lot of time, was in 2 situations:
- Using Claude to help me architect a secured deployment for MCP hosting with our our existing Azure tenant to allow more integrations with some of our products while retaining control of usage and access, down to the very requirements for network isolation to keep it internal, products we can use and will use, and everything in between. I am of course vetting everything it said, but so far it has been 100% dead on.
- User account compromise - Fable and Opus , the amount of audit logs I have to pull and review, Claude made quick work of it, parsing, grouping, creating accurate timelines of events, confirming my audits I pulled from specific queries, including some items I missed in one audit KeyQL query I was using.

What often can take me several days (when other things come up of course), took me 2.5 days exactly and that was me being overly meticulous myself not wanting to miss anything!


"Please fix this excel file for me" - while not including what is wrong or what they want fixed, then getting upset when it doesn't know what to do, or just goes off and does something different.

This is due to the marketing and over hyping of the product, aswell as some people not wanting to learn how to prompt effectively.

We have some Teams channels for CoPilot and Claude, and I am always posting guides on usage, better prompting and how to try and get the most of the tools, but the questions some people still ask...

View attachment 829342

Click to expand...

I would say it's more like a higher-level language. Because I've seen the same type of cognitive decline as well throughout my career and technology has lowered the bar of entry.

For example, Python vs C++. You can get a lot more done in Python than C++, and it has a ton of convenient features that that don't exist in C++. And it simplifies programming so that more people can do it and you don't have to worry about the myriads of intricacies of C++ which can lead to bugs. At the end of the day though, there's stuff C++ can do that Python just cannot. But for most people, the tradeoff of significantly longer development time is not worth it.

It's like the indie game scene. Today, many indie games come out which require hundreds of megabytes of storage and RAM, emulating an NES game which ran in 30KB. And the game is 1000x slower. That just doesn't matter for most people when your hard drive is over a TB in size and the initial time is 50 µs. It's like, I remember when MS Word took 6 minutes to even open compared to Word today which opens in a couple seconds and has vastly more features. Hardware improvements have allowed this tradeoff. For most projects, you probably don't need the technical knowledge.

The downside is, if you do need the technical knowledge. We're trading the removal of technical debt for an ever-increasing cognitive debt. As the other thread about NYT article about the studies of cognitive decline pointed out, it's not necessarily that the LLMs make us worse, rather, it's when you don't learn the fundamentals to begin with. AI there is like giving kids a calculator for math, but they've never learned how to add, subtract, multiply and divide in the first place. The problem here is that's becoming more pervasive in society. And companies are there to make money, expecting people to already know the fundamentals. Because at the end of the day, as good as LLMs are, reading code doesn't give you the same level of understanding as writing code. And now we're removing the latter while expecting people to do 100x as much reading.

philb2 said:

retarded ???

Click to expand...

Yes. My engineer is a functional retard. He is also a good friend and good at what he does. His parents didnt let him use the computer as a kid, so he is retarded at computer stuff.