Content
@
https://warpcast.com/~/channel/six
0 reply
0 recast
0 reaction
shazow
@shazow.eth
Long LSP and treesitter: they are *still* underrated in how impactful they are to the present and future of editor tooling. We're just starting to scratch the surface on what we can build with them. Here's things we haven't even done yet: - We can build jq that works with any language works with treesitter, not just JSON. - We can make a tool that generates control flow diagrams for any language. - LLMs can use TS grammars to tightly constrain valid output, vastly reducing training costs and possibly model sizes. - We can make plugins that augments tooling for any language that works with any editor (not just programming!), a lot of AI-assistants already use hooks like this. - Agents can interface with language structures at a higher level (AST) rather than sending text diffs back and forth. These two technologies began a golden age of editor tooling, and we're just getting started. https://x.com/shazow/status/1406289893895122951
5 replies
2 recasts
17 reactions
mindtree
@mindtrizzl.eth
Big agree for the majority of cases! My only qualm with both (treesitter especially) is that they tend to fall short for languages pushing the cutting edge with dependent types (like Idris, Agda, Coq). So editors like Helix that only support treesitter and LSP are ruled out for me.
1 reply
0 recast
0 reaction
shazow
@shazow.eth
Is there a constraint on how treesitter was designed? Or is it just a shitty implementation of the AST generator? In principle, it supports any program that produces an AST (doesn't have to be grammar-defined), so I can't imagine that it's fundamentally constrained.
1 reply
0 recast
0 reaction
mindtree
@mindtrizzl.eth
My limited understanding of the issue is that in dependent-type systems, the semantic highlighting is type-directed, and that you can't really achieve proper semantic highlighting with just a grammar. You can still make roughly-nice-looking syntax highlighting, but it won't always correctly distinguish types and values.
1 reply
0 recast
0 reaction
shazow
@shazow.eth
Sounds like a poor implementation of the AST producer. I don't think that's a limitation of TS itself, it works with anything that produces an AST, it doesn't need to originate from a grammar even. A lot of languages start from a grammar because there's an easy to use generator from that, but it's not a requirement.
1 reply
0 recast
0 reaction
mindtree
@mindtrizzl.eth
I think for dependently typed languages, you'd basically need the whole elaborator which is a large chunk of the compiler (idris' vim plugins work by using the compiler directly), but it sounds like maybe that's fine from what you're saying! I've only ever seen tree-sitter implementations that just use the JS grammar declaration thing to generate C code. I must be out of touch with what's actually possible!
0 reply
0 recast
1 reaction
