Akhil's Notes

Two tips to level-up ChatGPT & friends

Here's two simple tricks that consistently get much better outputs from AI chats. These take a bit more effort upfront, but the results are worth it. It's really changed how I use these models. And I think these could become standard practice, particularly the second technique which could change how we chat with AI entirely.

Note - From my testing, these techniques are most useful for thinking through ideas in a conversation or creating detailed documents.

Tip 1: Context First, Questions Later

Before getting to specific questions, it's important to set the scene. This helps the model understand your specific situation, which leads to more relevant and accurate responses. I find this most useful when you're trying to go deeper on a topic.

Add more detail than you think you need in your first message. If you're unsure about what to include, try using the Who/What/Why/How framework.

You may find the prompt ends up longer than you expected, upwards of 200 words. And that's fine, as long as it's all relevant information. The model will use this information in the rest of the chat.

After an lengthy first prompt, ask the model if it needs to know anything else about the topic. This will highlight things you've missed out or are unclear on. It'll usually ask for clarification or raise some tangential topics that it thinks are relevant. After you respond to these points, ask again if there's anything that's unclear. Repeat this process until the model is 'satisfied' - then ask your 'real' questions. The responses will be much more specific and relevant to what you're looking for.

Tip 2: Prompt-to-prompt

Improve basic prompts into high-quality instructions using an 'intermediary' chat. I find this two-stage approach produces better results than direct prompting, especially when you're trying to generate a detailed document.

Step-by-step:

You can use multiple chat windows with the same model for Chat 1 and Chat 2, which is simple enough with ChatGPT. I prefer to use two different AI models. It feels like each is better suited to different tasks. Pairing a model that has a more conversational style with another more advanced 'reasoning' model seems to work well. Claude Sonnet 3.5 and OpenAI o1-preview are what I've been using. I find Sonnet 3.5 gives a more natural feeling conversation, which gives a better user experience when having that initial back-and-forth. o1-preview, on the other hand, is difficult to converse with because of its reasoning; the results take significantly longer because they are a lot more detailed.

Additional thoughts

Context First, Questions Later

The biggest frustration when most people chat with an LLM is misalignment. Results don't meet expectations. I think this is why many people feel disenchanted, especially when using these at work. They're fine at asking for ideas for places to vacation; they're worse at preparing a brief for an upcoming marketing webinar.

I like to think of these LLMs as incredibly smart interns. Asking an intern simple tasks will usually give you decent results, especially if little research is needed or the format isn't important. Asking the same intern, no matter how qualified, to undertake a larger task with a simple brief will lead to a worse result. This is because you have a lot more context and understanding than you're sharing with the intern. Working with LLMs is very similar.

To use a different analogy, it reminds me of 'implicit knowledge' in UX design. A UX can feel complicated or overwhelming when an interface implies a lot of an understanding about how a system behaves. Using these tools without that knowledge can be challenging. Instead of getting frustrated, the LLM just returns a result with a series of assumptions to fill in the blanks.

Prompting-to-prompt

I believe this approach will become a standardized way to improve results. Most users are terrible at prompting, and this is the easiest way to increase quality. Similar to how Chain-of-Thought helps models break down complex reasoning into steps, prompting-to-prompt breaks down the user's request into a more effective format. I expect this will become how all prompts are handled in the future - our initial requests will be automatically refined by a specialized model before being passed to the primary model for processing. This will happen behind the scenes, without users knowing or intervening. Advanced reasoning models like o1-preview and tools like Replit Agent may already be doing something similar to improve responses.

Update - We may see this 'LLM intermediary' sooner than later. I thought the main use-case of prompting-to-prompt was to improve output quality, it could also be used to review or censor user input. Recently, I saw this simple but effective approach to jail-breaking LLMs, which intentionally obfuscates malicious prompts. Reducing quality or adding errors to input appears to bypass hardcoded safeguards. The model seems to correct these issues before responding. Without a reviewer checking user input against a terms of service, it's likely users will find more of these these workarounds. The obvious knock-on implication of an intermediary review is an increase in cost and time per response. If this happens, we'll see questions raised about censorship and the biases of the model creators.

Wrapping up

Adding these techniques to how you prompt does take a little more time and effort, but they've improved the quality of my outputs significantly. Context First helps avoid misalignment by limiting the number of assumptions the model is making on your behalf. Prompting-to-prompt helps prepare better instructions for more detailed outputs.

If you try these out or have other tricks to get better results - let me know!

#ai #writing