It's way better than the Google MD problem that previously existed.
I had Claude analyze a blood panel and provide potential diagnoses, medications, dosing and contraindications for a fairly esoteric condition. It gave me the questions to ask a doctor (being completely forthright that I'm yet another layman regurgitating AI output) who agreed with the approach and even admitted that he hadn't seen a case study it referenced but that he wish he knew about sooner.
Will doctors still be assmad that years of their study is now democratized? Probably. I am (or was) a software engineer, welcome to the club.
One of the most satisfying and useful applications for AI is handing it the URL for a recipe (replete with 8 paragraphs of useless introductory text, 2 paragraphs dedicated to how to refrigerate the meal, 9 inline advertisements and a mid-scroll ad) and having it return the actual recipe in Markdown in an Obsidian folder.
Surprisingly, a lot of these recipe websites (even the worst offenders for fluff and ads) have a clean JSON API for getting the recipe directly. I self-host a Mealie instance (https://docs.mealie.io/) which has an "import from URL" feature that uses this. It has rarely if ever failed me.
I've been maintaining a personal calorie tracker (keeps me entering), and using claude for entering meals has been the perfect experience.
One of my favorite features has been giving a recipe url and having it figure out the calories for that recipe with line-by-line entries for each ingredient. Then it keeps the extracted recipe in markdown format for the next time I want to make it.
Substitutions are also great - "I'm making [such and such recipe], but with half the sugar."
One of the others has been "I had a meal a couple weeks ago at [x restaurant], can you enter that again for right now?" Having the long-term context for entry has been excellent. And a related one, where I upload a photo of the menu, and say which menu items I'm eating and it puts together an estimate for the calorie count.
I lean heavily on a long auto-compacting claude code session for all of this. I have a UI for graphs, history, and simple entries, but claude's been my favorite means of entry by far
The other day I did that with a TikTok video! It was infuriatingly vague, so I extracted the mp4 using yt-dlp and fed the video file into Gemini Flash and told it to fill in the missing information and give me back a proper recipe.
Remember in 2023 when people thought "prompt engineering" would be the new software engineering and invested tons of time into learning CoT, ReAct, thread-of-thoughts, etc?
Those were mostly obviated by reasoning models and harness updates by 2024.
It seems pointless to invest energy into the latest/greatest AI technique or framework when they're going to either be absorbed or replaced on a 3 month cycle.
Isn't it clear that some people are better at working with/prompting LLMs than other people? Or is the idea that what you write to them and how you use them doesn't matter, it's all up to the model/harness? To me this seems clear, so then clearly this is a skill, which typically is called "prompt engineering". Specifically CoT or the other things you mention wasn't referred to as "prompt engineering" as far as I know, that skill is more about how you communicate with the LLMs and how you use them, rather than what specific processes/workflows/technologies you use.
I actually think that good prompting MOSTLY comes from good writing skills in general. Being able to more clearly state things to an agent, knowing what pieces of context are entirely unnecessary and which are important, having a larger vocabulary helps too.
Of course, there are other areas that can improve model output (Direction rather than open-ended assistance requests, using keywords + plugins that help, the "your output should include: " style prompting).
A few of us run almost the same exact setup at my shop (Base Claude Code w/ SuperPowers + a context repository) and the models are somewhat unhelpful to some, and give meaningful output to others. The only correlation I notice is that their prompts are no-good. Not from a meta "prompt" engineering standpoint, but from a general English 101 standpoint.
"dudde no i wanted the function to return 3 things. not like that. do it again"
VS something like
"Modify the "renderThreeVars()" function signature to accept another variable called "z" and add it to the return statement at line 64."
I ask it all the time about whether X is feasible, how we can get started on Y, and to investigate issue Z.
It is working great for me in a >100k LOC project.
Perhaps this works less well with weaker models. I suspect the people who say Qwen 3.6 27B is working well, are using prompts like "modify the renderThreeVars() function in rendering.py".
For example, if I tell it that I want my app translated, it can plan for me what the recommended options are in my framework, what languages I should target for my app, and come up with a skill for a repeatable workflow.
I like the fact that it ends up being a lifecycle. I know it can have different entry points based on what you ask, and which skill you trigger first, but it inherently is chaining together skills. There also seems to be logic built into it such that if the ask is small, but you've still triggered the skill with "brainstorm", it will make judgements like "want me to skip phase X and go straight to implementation?"
I've noticed among my coworkers that we all have different amounts of trust we're willing to give the agent. That seems to manifest into some people only asking questions about the existing code, but never writing anything new with it. Others are willing to do limited targeted changes with the agent, but are unwilling to do things like let it make commits, or connect MCP servers, or really do anything that isnt fully understood by the human before setting the agent loose. Then I find myself, who has dove headlong into it all. I have skills that use MCP servers to check for pull requests, and give me summaries to give me more context for code reviews. I update Jira tickets in batches of 50+. I develop complete features exclusively through prompting the agent to do everything. I know I still have the responsibility to understand it at the same level as if I wrote it my self, and defend it and debug it.
I can easily see that superpowers would be wasted on most of my coworkers, simply because the benefits compound with the complexity of my ask. My coworkers aren't willing to hand off enough control to receive the benefits of superpowers.
I am in the completely same boat as you and completely agree that you need to “give into it” more to get the most out of superpowers. I gain so much confidence out of the actual process and all of the adversarial reviews, re-reviews, refinement, etc.
Honestly, within the last few days I've gone back to a mostly vanilla installation of Claude with a handful of directives in AGENTS.md. Depending on what I'm working on, I don't feel I need the extra plugins or a context repository. (It DOES feel at times like it blows through tokens for no reason).
Been working on some things that require more targeted, smaller scale changes and the base models do perfectly fine when given good instructions.
Superpowers and Compound Engineering are the two that I hear about around our team, both seem "fine" if you're into the fully agentic engineering "big change" future technology stuff.
From some very quick tests, I get the impression that having good grammar, punctuation etc. is really not important (although I try to do it anyway because I'm accustomed to trying to do it), but clarity and precision definitely are. And of course, if you have unknown unknowns, they do need to get figured out before progress is possible. But with the right mindset, that just means you need an extra turn or two, not that you're going to end up at a dead end. (... I guess that counts as a pun?)
I will often do "speak a long rambling set of ideas and ask the LLM to summarize it into a prompt or spec -> manually refine -> drop refined prompt into fresh conversation"
As part of a previous job I needed to audit internal AI usage from a largely non-technical employee population.
The prompts were, predictably, really bad. Broken English, sentence fragments, vague requests, lack of context. Yet somehow, the users always got the answer they were looking for. It might have taken a few extra turns with questions from the model, but the end result was the same.
It's humbling, but a flowery, carefully crafted prompt is at best slightly more efficient than a "CAN A DOG BE EATIN SUN FLOWER SEED?" peasant prompt.
You call that a peasant prompt, but it's actually almost perfect. Couple notes, but it's 95% of the way there. "Can a dog eat sunflower seed" is probably the perfect version, just 1 extraneous word in this version.
Unless the user wanted to know if a cat could eat sunflower seed or something.
To be pedantic, these types of prompts work best when in first-person/roleplaying. So the "perfect" prompt here would be something like, "I'm a dog and I just ate 20 grams of salted sunflower seeds with the shell on. Because I'm a dog I sometimes eat things without thinking about it. I'm worried about the short and long-term physiological consequences of what I've just done..."
I copied your prompt verbatim to ChatGPT and to Google.
ChatGPT kept the charade for all of one sentence. Then it dropped to talking about "your dog" the rest of the way. It even starts the final paragraph with "if, instead, you mean you (a human) ate them...", and finishes with the question "is this about an actual dog or yourself?"
Google did consistently refer to me as a dog, but its entire focus was on the steps "your human" should take, no advice for the dog itself.
In both cases, it looks like the first person roleplay was entirely inconsequential for the usefulness of the output. I think the current gen AIs have outgrown this trick and you can safely forget about it.
Claude gave me a very factual answer. Listing off the potential effects, not once referring to me being a dog or human. Then it ended with “ Call a vet if you see repeated vomiting, no bowel movement for over 24 hours with straining, a hunched or painful belly, lethargy, or blood in stool. Otherwise monitor and carry on being a dog.”
Doesn't work. ChatGPT realizes you're roleplaying and still treats you like a human. It even generates two alternative paragraphs of advice, one labeled "if we're staying in character" and one labeled "if this actually happened to a real dog". The latter is of noticably higher quality, too.
"dog sunflower seeds" is all the context an LLM needs, any other words are extraneous.
They don't need to be told that you're asking about the safety of eating them, because they can infer that based on the fact that a very large percentage of any text linking dogs to sunflower seeds is obviously going to be about the safety of the dog eating them.
Even pre-LLM that would have been a perfectly sufficient google search for the same information.
I think some people who are better at "prompting" even without domain knowledge could be better at getting LLM agents to produce good results than people with good domain knowledge but without the skills to prompt well. Just a hypothesis though, would be fun to try it out for real sometime :)
Pit two people with different "prompt engineering" familiarity against each other in real-time, with the same goal, see who builds the best thing, judged by other human experts.
RL has basically killed prompt engineering. You still need to provide the right context and process, but how you communicate with them beyond that is no longer so important.
> Isn't it clear that some people are better at working with/prompting LLMs than other people?
Sure, but I think it's essentially just that people who are better at traditional non-agentic software engineering are better at agentic software engineering. The only exception would be individuals who avoid agentic coding due to skepticism, hostility, or lack of opportunity.
I feel the same way about "prompt engineering" as I feel regarding the term "parkour" - you know, running and jumping on stuff.
Are people really putting on their resumes that they are capable of reading and writing and appropriately defining and limiting context? That's all prompt engineering is - it's being able to communicate effectively and elucidate your objectives.
Congratulations to all you English majors out there, you're about to make $350K/year.
Personally I don't, but why not? People aren't embarrassed to put their language skills ("be able to communicate in this specific language" - not special, it's just another language?), their leadership skills ("effective business communication" - big deal) or that they are a people-person ("can talk with others" - most people can do this) on their resume.
> Are people really putting on their resumes that they are capable of reading and writing and appropriately defining and limiting context?
People put whatever buzzwords will get them through initial screening. My resume contains tons of banal shit like agile, automated testing, Linux, AI (since 2018), and design patterns.
> Or is the idea that what you write to them and how you use them doesn't matter, it's all up to the model/harness?
Yes, it is. Source: had models inplement complex things from scratch and bullshit regardless of whether it was a one-line prompt or a detailed "SOTA witchcraft magic spells that are guaranteed to work"
It's not really a skill. The models are at this point smarter than you are, so the idea that you can prompt them "better" is laughable really when discussing frontier models.
It's like imagining you could "prompt" Richard Feynman to be smarter at Physics.
That is, for 99.9% of engineers, if you want the model to do a code review of your project, the best solution is to just ask Fable, "Hey Fable, do a code review of this project." Throwing in extra text like "think like a senior engineer", "ensure you focus on DRY principles, KISS, self documenting code, etc", doesn't make a difference.
These sorts of tricks used to work with dumber models, but now, like I said before, it's like thinking you can prompt Linus Torvalds into writing better C than he already can do.
> it's like thinking you can prompt Linus Torvalds into writing better C++ than he already can do.
Linus Torvalds, the inventor (and beloved dictator) of Linux, has always been quite harsh about C++ and why he rejects it for Linux kernel development. He’s not just been very vocal about it, but also brought up some arguments against the use of C++ that are worth reviewing in detail.
The models are beyond expert level in many areas at this point.
Do you really believe that adding extra junk to your prompt is going to make the model write code better than it does already?
Again, imagine going to Terrence Tao and "prompting" him to get better at Maths, do you think you can do it? What prompt would you give to him to make him produce better maths. Unless you're already a world-leading Mathematician I think you would find it hard.
Models are not rational thinking brains, so the point is moot.
Extra prompt text isn’t to make the model smarter, it’s to pull attention towards what you want. “Do code review” is far different from “Make sure changes align with existing architecture” or “follow these enterprise standards XYZ”. The model just acts as an average of its training data, which may or may not align with your goals.
I find this very much untrue in my practice and my testing, where fable prompted to do a review found surface level issues, while prompting along the lines of "assume it's wrong, prove it's correct" found much more in depth and real issues.
Understanding how to write good prompts (prompt engineering) is still very much a relevant skill if you want to effectively use LLMs. Harnesses aren't magic.
> Remember in 2023 when people thought "prompt engineering" would be the new software engineering and invested tons of time into learning CoT, ReAct, thread-of-thoughts, etc?
Prompt engineering was a GPT-3 era term, which couldn't understand instructions. Then ChatGPT came out in late 2022, which made actual prompt engineering superfluous.
No, people tried to sell “prompt engineering” services well beyond that — especially for image generating AIs, which where believed to need extra instructions like “professional lighting”, a name of a camera model, lens parameters, etc.
I still believe in writing good prompts or good instructions. Bad prompts can sometimes blow up the bill. A poorly written spec can waste a lot of tokens
The globalists had no problem selling out our manufacturing to China, now they are crying because China can provide a cheaper service.
The globalists of old would happily hand AI dominance to China in exchange for a decade or two of short-term gains. The fact that protectionist measures are even being considered shows that the globalists who sold out manufacturing to China aren't at the wheel anymore.
They are. Selling out was done with the expectation that "the West" will outsource the basics, but will have hands freed to "reach singularity" and will keep the braids of all the technological chains. Now the "reaching singularity" itself is being lost, so everyone suddenly understands that there is nothing left.
US banks would be prohibited from processing payments for those model providers, and those model providers would update their EULAs to prohibit American-usage.
Look at how online poker was "banned" in 2011. It was and always has been legal to play online poker in the US, they just made it illegal to accept wire payments for online gambling which destroyed the American market for it.
So many comments in this thread miss the point; an American "ban" on Chinese open-weights is not aiming to make individual usage impossible and wouldn't be accomplished by throwing people in jail. The point is to make their usage difficult and uncertain enough that large companies will opt for American alternatives.
Zitron thinks capex is a liability that needs to be paid off in a year instead of a long-standing asset.
Similarly he thinks that an investment into an AI startup is also a loan that the startup needs to pay back out of their own revenue, instead of a share of a company that will IPO at a higher valuation.
Basically his doomerism is a byproduct of financial illiteracy.
The Lambda-SQS poller will slow down in response to the lambda's error rate for this exact reason.
Note that this can cause issues: say you have a time sensitive application that receives a batch of "bad" messages which cause failed lambda invocations. The poller will slow down and the throughput will drop drastically, even though your intention might be for the lambda to continue processing at the same rate and power through the bad messages.
This behavior can be disabled with a support request.
You're confusing the discussion by trying to force the "leader" terminology. That term does not appear in the BTC whitepaper and the protocol's approach to consensus is different than a traditional leader elected system.
There is no "voting" and no "leader" except in the most abstract sense and I'm not sure why you're so determined to use those terms.
I had Claude analyze a blood panel and provide potential diagnoses, medications, dosing and contraindications for a fairly esoteric condition. It gave me the questions to ask a doctor (being completely forthright that I'm yet another layman regurgitating AI output) who agreed with the approach and even admitted that he hadn't seen a case study it referenced but that he wish he knew about sooner.
Will doctors still be assmad that years of their study is now democratized? Probably. I am (or was) a software engineer, welcome to the club.