An experimental TypeSpec syntax for Lexicon

First-class support for External Lexicons #5

closed
opened by danabra.mov edited

The current state of externals is documented here:

https://tangled.org/@danabra.mov/typelex/blob/main/DOCS.md#external-stubs

Currently, if you want to refer to a Lexicon (which isn't written in Typelex), you can stub it out like

// Empty stub (like .d.ts in TypeScript)
namespace com.atproto.label.defs {
  model SelfLabels { }
}

It's enough to have the namespaces and the model names. (And maybe @token decorators for tokens?) You don't need to fill in the entire thing.

Still, these stubs are tedious to create by hand. Something feels off about the workflow. You also don't actually want them to appear as the output JSON. Instead, you want the original Lexicon JSON for them in your lexicons/ folder.

This is very similar to the concept of "externals" in JavaScript bundlers.

I think an ideal workflow might look closer to this:

  • You check in some external lexicons into lexicons/ manually. Maybe you even use some tool that does it by nsid via Lexicon resolution. For example, https://jsr.io/@lpm/cli does this via lexicons.json.
  • Typelex automatically generates a file with .tsp stubs for those Lexicons from JSON. Maybe this file is gitignore'd. It doesn't need to actually translate them because simply declaring each namespace per JSON file and a model per each of its defs should be sufficient. This makes them appear in IntelliSense, etc.
  • Typelex also knows to not include those external Lexicons into the output. I.e. it shouldn't write them to lexicons/ since they are external.

Key questions:

  • This requires Typelex itself to have this logic somewhere. So it's maybe no longer just an emitter but a higher-level tool. A CLI (https://tangled.org/@danabra.mov/typelex/issues/4) seems like an obvious place to do that.
  • Whether externals are configured or inferred. How do you know something is an external?
    • If it's explicit, where is it configured? Is this Typelex configuration or something more general? Should Typelex adopt lexicons.json from https://jsr.io/@lpm/cli?
    • If it's inferred, what would it be inferred based on? We probably shouldn't infer it based on which namespaces are "missing" since those could just be typos. It's also not necessarily the case that you only work in one primary namespace. I think inferring might just be too confusing.
    • If .tsp externs get generated, how do they become available to the compiler? Is the generated file somewhere in your repo? Where do we put it? Is it continuously updated or is there a command to do that? It can't just be done at compilation because you also want the LSP to be aware of external lexicons. Maybe it's a command to the CLI.

I.e. it shouldn't write them to lexicons/ since they are external.

This should be easy if the emitter knows they are externals. Then it can know just not to emit them. Maybe there could be an @external decorator to mark them. Then the stubs generated by the CLI could have that decorator.

this works now

https://bsky.app/profile/danabra.mov/post/3m2ujpad3zk22
https://typelex.org/#install

it doesn't rely on lexicons.json, instead we just ask you which namespace you'd like to write in typelex.

whatever you specify gets written to your package.json. all other namespaces will be turned into externals

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:fpruhuo22xkm5o7ttr2ktxdo/sh.tangled.repo.issue/3m2otiowkr722