Content
@
0 reply
0 recast
0 reaction
blobs
@blobs
i'm figuring out how to train a LLM and documenting each step in this blog series. for anyone who is also curious, here is part 1: https://michaelhly.com/posts/train-llm-one
4 replies
5 recasts
25 reactions
vincent
@pixel
blobs, are you doing AI for blobs (the product) or is it exploration? also: what limit should i reach before i consider tuning Llama? what does it look like?
1 reply
0 recast
0 reaction
blobs
@blobs
a). just exploration b). i'm not sure what you mean by limit. but hugging face has pretrained llamas that you can grab off the shelf: https://huggingface.co/meta-llama my belief is that you should tune a model to find a fit for your dataset ... otherwise you probably don't need to ...
1 reply
0 recast
0 reaction
vincent
@pixel
limit means: gpt4 is pretty great, when should i consider having my own models, i want to play with it, but i don't have a hair-on-fire problem i want to solve with some custom llm, no strong motivation
1 reply
0 recast
0 reaction
blobs
@blobs
yes. so this is the hardest part — defining the output/results you want the model to tune towards yes for generative use cases, it's hard to beat openai. i believe people usually tune for niche/task-specific use cases.
2 replies
0 recast
0 reaction
Jason — ⚪↑🎩 🧾
@jachian
Another I would add is making sure it’s not a problem that can’t already be solved by improving prompts I’ve held off on fine tuning models because I don’t have a use case that’s specifically terrible with GPT-4
1 reply
0 recast
0 reaction
Jason — ⚪↑🎩 🧾
@jachian
Visual example is that when Dreambooth first came out, the land grab came for the AI Avatars for adults This is because the base models had holes in their dataset wrt young children and pets that weren’t dogs or cats Even this will be abstracted away with time by MJ improvement
0 reply
0 recast
1 reaction
