11 — UI (2D GUI, particles, MSDF text, 3D GUI)

Subsystem diagram: ui.html · Code traces: 2D GUI · Particles · MSDF text · 3D GUI runtime · 3D GUI authoring

Engine code lives under packages/engine/src/ui/ (2D + 3D GUI + MSDF text) and packages/engine/src/subsystems/particles.ts (particle JSON loading).

Overview

Three related but distinct UI/media paths share the same loader pattern: queue async work during the entity pass, settle in FinalizeLevel. 3D GUI is different — it is fully synchronous but deferred to a post-pass because panels must exist before child controls and click targets must resolve through the finished Level.

flowchart LR
  subgraph export [Export]
    GUI_JSON[GUI .json]
    PART_JSON[Particle .json]
    MSDF_FONT[MSDF font JSON + PNG]
    GUI3D[GUI3D_* components]
  end
  subgraph manifest [level.scene.json]
    M_GUI[GUI component]
    M_PART[PARTICLE component]
    M_MSDF[MSDF_TEXT component]
    M_3D[GUI3D_* types]
  end
  subgraph load [Runtime load]
    Apply[ApplyComponents]
    Final[FinalizeLevel]
    GUI2D[ApplyGui]
    PART[ApplyParticles]
    PARTWIRE[WireParticleEmitterTracking]
    MSDF[ApplyMsdfText]
    WIRE[WireMsdfTextRendering]
    BUILD[BuildGui3DControls]
  end
  GUI_JSON --> M_GUI
  PART_JSON --> M_PART
  MSDF_FONT --> M_MSDF
  GUI3D --> M_3D
  M_GUI --> Apply --> GUI2D --> Final
  M_PART --> Apply --> PART --> Final --> PARTWIRE
  M_MSDF --> Apply --> MSDF --> Final --> WIRE
  M_3D --> Apply --> BUILD --> Final

Per-frame observers: empty-node particle emitters register onBeforeRenderObservable with insertFirst=true (before Level.RunFrame). MSDF labels draw on onAfterRenderObservable — after the main GPU pass. See 02 — Runtime Basics · runtime-loop.html.

2D GUI (GUI component)

Authored in Blender with a file picker pointing at a layout saved by Babylon's GUI Editor. Export copies the JSON into gui/ via copy_asset (stable sanitized name; re-export overwrites).

Runtime: ui/gui2d.ts → ApplyGui → entity.guiTextures + RegisterAttachment. Lookup: entity.GetAttachment("GUI")?.texture, entity.GetGui("hud") (file stem) → texture.getControlByName(...).

Particles (PARTICLE component)

Authored with a file picker for a system saved by Babylon's Particle Editor or the Node Particle Editor (.json with BABYLON.NodeParticleSystemSet).

Blender authoring

After picking a particle JSON, click Scan Textures to populate one row per ParticleTextureSourceBlock. Re-scanning preserves existing image picks by block id. Each Textures row shows the block label and current JSON URL when present:

Export calls export/particles.py → export_particle_system: copies the JSON, copies picked texture images, then patches ParticleTextureSourceBlock.url in the exported copy by block id (the manifest still only references particles/<stem>.json). Validation warns when a scanned slot has no URL in the JSON and no image was picked.

Runtime load

subsystems/particles.ts → LoadParticleSystems (legacy ParticleSystem / GPU, or NodeParticleSystemSet.buildAsync). Before build, ResolveNodeParticleSetTextureUrls rewrites bare filenames in the JSON to absolute URLs beside the file (rootUrl). Options: GPU when supported, autoStart, attachToEntity, optional capacity override.

When attachToEntity is on:

Already-spawned particles still simulate in world space unless the particle file sets isLocal: true (local / emitter space in the Particle Editor).

Lookup: entity.GetAttachment("PARTICLE")?.system, entity.GetParticles("fire").

MSDF text (MSDF_TEXT component)

Authored in Blender with paragraph Text, a BMFont Font JSON, and the paired glyph atlas Font Texture PNG. Generate assets with msdf-bmfont or msdfgen; export copies both into fonts/ via copy_asset.

Runtime: ui/msdfText.ts → FontAsset (cached per scene) + TextRenderer.CreateTextRendererAsync → parent to entity.node → addParagraph with authored align/wrap/stroke/billboard options → entity.textRenderers + RegisterAttachment. After SettleTasks, WireMsdfTextRendering hooks scene.onAfterRenderObservable to call renderer.render(view, projection) for every label (level.msdfTextManager, disposed with the level).

Lookup: entity.GetAttachment("MSDF_TEXT")?.renderer, entity.GetTextRenderer("roboto-regular") (JSON file stem). Dynamic copy: clearParagraphs() then addParagraph(...). Depends on @babylonjs/addons (peer dep, like @babylonjs/gui).

3D GUI (GUI3D_* components)

No external editor — Blender is the layout tool. One component per Babylon control type:

Hierarchy (Blender parenting)

flowchart TB
  Panel[Panel empty\nGUI3D_CYLINDER]
  B1[Button child\nGUI3D_HOLO]
  B2[Button child\nGUI3D_BUTTON]
  Panel --> B1
  Panel --> B2
  Panel -->|BuildPanels| BP[Babylon panel\nlinkToTransformNode]
  B1 -->|parentPanel.addControl| BC1[Control in panel layout]
  B2 -->|parentPanel.addControl| BC2[Control in panel layout]
  BP --> BC1
  BP --> BC2

Parent button objects under the panel empty (Ctrl+P). Child transforms express membership only — the panel arranges controls at runtime. Standalone buttons (no panel parent) are added to the manager root and linkToTransformNode'd to follow their anchor. GUI3D_MESH keeps the mesh's own world placement.

Click events

Each button's On Click Events list (target + message) mirrors trigger collider rows. At runtime WireClickEvents subscribes to onPointerClickObservable and calls target.SendMessage(message, buttonEntity) — behaviors handle it via OnMessage, same as trigger volumes.

Build order (critical)

sequenceDiagram
  participant FL as FinalizeLevel
  participant B as BuildGui3DControls
  participant P as CreateGui3DPanel
  participant C as CreateGui3DControl
  participant A as ApplyControlContent
  participant W as WireClickEvents
  FL->>B: after BuildConstraints
  B->>P: panels: addControl then linkToTransformNode
  B->>C: controls: addControl to panel or manager
  C->>A: content AFTER addControl
  A->>W: click → OnMessage

Babylon silently drops button content set before addControl — see ui/gui3d/controls.ts.

Runtime API

• level.gui3DManager — shared manager (disposed with level)
• entity.controls3D — every control/panel owned by this entity
• entity.GetAttachmentsOfType("GUI3D_BUTTON") — component rows with control ref
• entity.GetControl3D("StartButton") — named after the Blender object

Loader integration

In the component registry (loader/componentRegistry.ts, per handler):

• GUI → context.guiTasks.push(ApplyGui(...))
• PARTICLE → context.particleTasks.push(ApplyParticles(...))
• MSDF_TEXT → context.msdfTextTasks.push(ApplyMsdfText(...))
• GUI3D_* → context.gui3dRegistrations.push({ entity, component, parentId })

In FinalizeLevel (LevelLoader.ts):

1. SettleTasks for audio, GUI, particle, and MSDF text promises (allSettled)
2. WireParticleEmitterTracking → level.particleEmitterManager (when any

empty-node emitters exist)

1. WireMsdfTextRendering → level.msdfTextManager (when any labels exist)
2. WireCollisionEvents
3. BuildConstraints
4. BuildGui3DControls → level.gui3DManager
5. level.Begin()

Blender add-on