AI img2mesh with LLMs: mesh from a reference image, authored as DSL

AI img2mesh through code: author DSL from an optional front reference, repair up to six compiles, silhouette-compare up to two times, then publish.

I found something while poking at modelmaker that I did not expect to work this cleanly.

Frontier language models can author a 3D character mesh in code from a reference image. Not by dumping triangles. By writing DSL syntax against a spec, inside a harness that compiles the source, repairs it, and when you attach a front full-body reference, renders the front view and vision-compares silhouette and proportions.

That is the discovery. Not a replacement for artists. Not a replacement for image-to-3D products. Not earth-shaking. A new capability that shows up when you harness next-token prediction correctly.

AI img2mesh, through code

Most AI img2mesh / image-to-3D tools take a picture and predict a mesh. That path is real and useful. This is not that path.

This is AI img2mesh where the model authors mesh source from the reference image. The artifact is a DSL source file. A harness compiles it, repairs it against a linter, and silhouette-compares the render to the reference. If you came here searching for AI img2mesh, that distinction is the whole post.

What modelmaker actually does

The entry point is a search box. Prompt in. Optional front-facing full-body reference image. Then:

1. Author writes DSL syntax against a spec (reference attached when present).
2. Server compiles and runs a repair loop, up to 6 rounds, feeding real linter output back. Still unclean after the cap means the job fails. No shipping a warning-laden “ready” entry.
3. If there was a reference: render front view, vision-compare silhouette and proportions, up to 2 fix passes.
4. Publish: glTF, turnaround sheet, animation strips, interactive viewer, catalog entry.

Every model is authored from scratch. No base mesh. No template. A bird request has to come out as a bird.

Typical wall clock is about 1.5 to 5 minutes per model, most of it authoring and repair. The backends I wire in are Grok CLI, Codex CLI, OpenRouter, and LM Studio. The CLIs do not compile anything themselves. Grok runs with tools denied for this job. Codex runs read-only. The server owns compilation and the repair loop either way.

A real repair trace from the README looks like this:

authoring          367 lines returned
  1 issue:         ERROR: expected a tuple like (x,y,z), got '(0.03,0,'
repair round 1     14 issues:  ring does not understand 'down'...
repair round 2      3 issues:  'chest_ruff' pins its ring frame nearly parallel...
repair round 3     compiles clean

Clean after one try is impressive. Six tries is fine if the final mesh is still good. The loop is the quality mechanism.

Why a DSL instead of raw glTF

The DSL is a text language for low-poly characters, rigs, and animations meant for LLM authorship.

The author only makes discrete, named, relative, symmetric decisions. Bone directions as words and degrees. Masses as cross-section rings. Left side written once inside a mirror block. The compiler owns vertices, skinning weights, normals, and winding. The design rule is that it is structurally hard to emit a non-manifold mesh or a disconnected rig.

That is why the DSL matters for this ability. Raw glTF is a terrible authoring surface for a language model. DSL syntax is faster to write than dumping glTF, and the source stays readable and repairable. That also makes it more shareable as UGC: you pass around a text file people can edit, not a binary blob nobody wants to touch.

Before geometry, the authoring prompt forces a four-line design plan: palette, proportion, silhouette, signature. It will not mark a model ready while the compiler still reports WARN or ERROR after repair.

How you score it: silhouette from angles

Shading lies. A lit turnaround hides bad mass with colour, occlusion, and gradients.

modelmaker judges shape the way the tooling does: flat silhouette, including thumbnail sizes. the silhouette script renders outline views and a thumb strip. The note in that script is blunt. Measured on this repo’s own models, the hero still reads at 24px as an anonymous humanoid, while a scorpion is identifiable immediately from raised tail and claws. Neither fact is obvious in a shaded sheet.

So the honest score for a reference run is:

• Does the front silhouette hold the reference proportions.
• Do side and three-quarter reads still look like the same creature.
• Did vision compare actually pull mismatches in, or did the model invent a generic member of the archetype.

Reference authoring also has a written discipline in the reference-image notes: enumerate exhaustively from crops, build selectively. About five to eight features earn geometry beyond the base body. Everything else is palette, material bands, or texture. If it would not change the shape at 32 pixels, it is not geometry.

What clears the bar, and what does not

On my runs, the setups that can author a mesh from a reference image are Claude Fable 5, Claude Opus 5, GPT Sol at xhigh, and Grok 4.6 at xhigh.

Weaker models cannot reason in 3D space the same way. The output is junk clumps. Compiling does not save them. They do not hold volume, joint direction, or the reference silhouette.

Local and mid-tier backends fail in more boring ways too. In the catalog I have LM Studio jobs that died on an 8192-token cap mid-author, or timed out after retries. That is a different failure mode from “wrote a horse-shaped mesh that survives checks,” and it is useful to keep separate.

The catalog already has successful non-reference and reference-style runs that show what “good” looks like numerically. A World of Warcraft human paladin authored via Codex landed at 1476 triangles, 19 bones, idle and walk, 23 authored checks passed, including tris under 3500, plantigrade feet near y=0, elbow/knee bend direction, and walk slide near zero. A tortoiseshell cat via Grok got 1224 triangles and mostly passed, and still failed one gait check: slide(hindpaw.l, walk) = 0.0818 against a < 0.03 budget. That is the point of the harness. Pretty still loses to a measured foot slide.

Ambient lints exist because silent geometry bugs used to compile clean and look plausible. Elbows that bend like knees. Arms lofted from the upperarm so the clavicle leaves a shoulder gap. Flat foot bones that make paddle feet. Those are in the harness mistake notes because they cost hours when the mesh looked fine.

The $40 horse

One frontier run made a horse that cost about $40 in API spend.

People online argued about whether that was too much for AI or too little for an artist. The quality confused the pricing intuition. Useful signal. Not a business model. Not proof that artists are obsolete. Proof that the output crossed the line where people stop treating it like autocomplete noise.

What I am actually claiming

Vibe exploration of LLM capabilities.

Next-token prediction keeps producing interesting knock-on effects when you put the model in a loop that can compile, lint, render, and look. Authoring a 3D mesh in code from a picture is one of those effects. Narrow. Specific. Only a few frontier setups clear it today. Worth writing down.

Not a new industry overnight. A discovery, and a harness to keep measuring it.