07 — Rendering: lights, cameras, shadows, scene look, node materials

Subsystem diagram: rendering.html · Code traces: Lights · Cameras · Shadows · Post-processing · Atmosphere · Node materials · Blender: NME authoring

Lights (subsystems/lights.ts)

Automatic — no component. The glb creates and places the light; ApplyBlenderLight copies the lamp's properties onto it. Blender inserts an orientation-correction node when exporting +Y-up, so the GUID node is an *ancestor* of the light — FindLightForNode walks the whole parent chain. POINT→Point, SUN→Directional, SPOT→Spot (AREA unsupported by glTF — validator warns). Color and intensity are copied 1:1 from the manifest (energy → light.intensity): point/spot power in watts, sun strength in W/m². When Atmosphere is on, PBR Sun Intensity (default) may override the sun to π — see Atmosphere below. SUN lamps export sunAngle (angular diameter, radians); when the lamp casts shadows, SetupShadows maps it to PCSS penumbra (Blender Angle 0°→0, 45°→1; clamped above 45°). Spot lamps export spotSize / spotBlend. No fallback light — a dark scene means add a lamp in Blender.

Punctual light budget (subsystems/clusteredLights.ts)

Large light rigs can exceed WebGL's per-shader uniform buffer block limit (often 12–16 blocks; Babylon uses one block per forward light plus scene/material/mesh data). Symptom: Unable to compile effect / _finalizePipelineContext in the browser console with a high LIGHTCOUNT define — not a Blender export bug.

The loader handles this automatically during FinalizeLevel, before SetupShadows and the first material compile. Authors do not toggle anything in Blender for normal use.

When it kicks in

After the entity pass, count enabled lights in scene.lights. When the count exceeds the budget (default 8), the loader tries to move eligible point and spot lights into a Babylon ClusteredLightContainer. Directional (sun) lights stay forward so shadow maps and PCSS penumbra keep working.

Three outcomes (level.punctualLightingMode)

On load, check the console for a summary line:

[bjs] punctual lighting: clustered 19 point/spot lights (2 regular scene lights remain; budget 8)

or forward-expanded (uniform buffers disabled; …). You can also read level.punctualLightingMode from gameplay code after LevelLoader.Load.

Clustering details and limits

• glTF punctual lights export with Light.FALLOFF_GLTF; clustered lighting only accepts FALLOFF_DEFAULT, so the loader converts falloff before addLight. The cluster writes inverse-square attenuation into its light-data texture — close to glTF, not bit-identical to forward.
• Clustered lights cannot cast shadows in Babylon. Keep shadow-casting on suns (and any punctual lights that must shadow) as regular forward lights; expect those to count toward the budget.
• IES profiles and projection textures on spots are not clusterable.
• Devices without WebGL2 float blending fall back to forward-expanded.

Overrides (optional)

Most scenes should leave these unset. Manifest scene block (no Blender panel yet):

"scene": {
  "clusterPunctualLights": true,
  "lightBudget": 8
}

LevelLoader options clusterPunctualLights and lightBudget override the manifest when set. Trace: entity pass applies lamps → ClusterPunctualLightsIfNeeded → SetupShadows.

Cameras (subsystems/cameras/)

Mirrors the lights design: the glb creates a faithful FreeCamera (FindCameraForNode walks parents the same way); ApplyBlenderCamera copies clip range and FOV/ortho mode; the exporter flags the scene's active camera → scene.activeCamera + level.activeCamera. Faithful playback by default — no controls attached; main.ts adds a fallback ArcRotate only when no camera shipped.

An opt-in CAMERA component overrides the type via BuildTypedCamera, which derives the new camera from the faithful one's world transform (so it starts exactly where Blender framed it), copies the lens, then disposes the original. Per-type builders: UNIVERSAL (free-fly + key scheme), ARC (orbit; optional target re-pivots in the second pass; setPosition(eye) keeps the Blender framing; optional Track Target drives the orbit pivot each frame via Level.AddUpdater), FOLLOW-ORBIT (Babylon FollowCamera; useBlenderTransform derives radius/height/rotationOffset via DeriveFollowFromPosition), FOLLOW-OFFSET (a UniversalCamera driven each frame by a Level.AddUpdater to hold the exported world offset), GEOSPATIAL (Babylon GeospatialCamera for map-like globe navigation — pan, zoom-to-cursor, tilt; the planet mesh must sit at world origin and Planet Radius must match its radius in scene units; DeriveGeospatialPose raycasts the exported eye/forward against that sphere to seed center / yaw / pitch / radius; optional min/max zoom and collision checking). Orbit Speed, Zoom Speed, and Pan Speed (manifest orbitSpeed / zoomSpeed / panSpeed, default 1.0) apply to ARC and GEOSPATIAL when controls are attached — implemented in speeds.ts (ApplyArcRotateControlSpeeds maps to Babylon sensibility / wheel precision; ApplyGeospatialControlSpeeds scales movement.rotationXSpeed / rotationYSpeed / panSpeed / zoomSpeed). Free-fly cameras (FREE/UNIVERSAL) also honor an optional Keep Upright toggle (lockRoll): LockCameraRoll bakes the world pose and detaches the glb camera from its orientation-correction parent (so yaw/pitch happen in world space), pins look-at to world up, and drops any residual view-axis roll each frame — keeping the camera level with the horizon. Targets resolve with entity references in the loader's second pass (pipeline step 5). GEOSPATIAL has no deferred target — controls (pointer, wheel, keyboard) attach when Attach Controls is on.

CAMERA component example (ArcRotate)

Orbit a moving target; speeds are multipliers over Babylon defaults:

{
  "type": "CAMERA",
  "cameraType": "ARC",
  "attachControl": true,
  "keys": { "scheme": "ARROWS", "up": "W", "down": "S", "left": "A", "right": "D" },
  "useBlenderTransform": true,
  "followMode": "OFFSET",
  "lockRoll": false,
  "speed": 1.0,
  "inertia": 0.9,
  "radius": 10.0,
  "lowerRadius": 0.0,
  "upperRadius": 0.0,
  "target": "<entity-guid>",
  "trackTarget": true,
  "orbitSpeed": 1.0,
  "zoomSpeed": 1.0,
  "panSpeed": 1.0,
  "distance": 10.0,
  "height": 4.0,
  "rotationOffset": 0.0
}

CAMERA component example (Geospatial)

Globe mesh centered at world origin; camera starts at the exported Blender pose:

{
  "type": "CAMERA",
  "cameraType": "GEOSPATIAL",
  "attachControl": true,
  "planetRadius": 1.0,
  "lowerRadius": 0.01,
  "upperRadius": 0,
  "checkCollisions": false,
  "orbitSpeed": 1.0,
  "zoomSpeed": 1.0,
  "panSpeed": 1.0
}

planetRadius must match the globe mesh radius. lowerRadius / upperRadius map to Babylon's limits.radiusMin / limits.radiusMax (0 = no limit). orbitSpeed / zoomSpeed / panSpeed tune pointer orbit, wheel zoom, and pan when controls are attached (1.0 = default). Runtime-only APIs such as flyToAsync remain on the live GeospatialCamera instance.

Large world rendering / floating origin

When coordinates are very large (geospatial globes, flight sims, open worlds), enable Large World Rendering in Blender's Babylon Scene › Rendering panel. Export writes scene.largeWorldRendering and optional scene.floatingOriginWorldRadius (Havok multi-region radius, default 100000).

The app must fetch the manifest before creating the Babylon engine — CreateLevelEngine sets useLargeWorldRendering; ResolveHavokPhysicsOptions passes the region radius to EnableHavokPhysics. See core/bootstrap.ts and apps/playground/src/main.ts. Meshes, behaviors, and the loader otherwise stay unchanged; Babylon offsets shader uniforms relative to the active camera and Havok simulates bodies in regional floating-origin worlds automatically.

Shadows (subsystems/shadows.ts)

Driven by each lamp's Cast Shadows toggle; one ShadowGenerator per casting light. Meshes always receive shadows; casting follows Blender Object Properties → Visibility → Ray Visibility → Shadow (visible_shadow) — when off, export stamps bjs_cast_shadows: 0 in glTF extras and SetupShadows skips that mesh as a caster (receive-only). Typical use: a huge sea floor that should catch train shadows but must not blow up the sun frustum. Per-light settings: filter (PCF / PCSS / Poisson / Blur ESM / hard), bias, normalBias, darkness, mapSize, frustum minZ/maxZ (set on the *light*; 0 = auto), frustumEdgeFalloff (directional/spot edge fade), and forceBackFaces (render only back faces into the shadow map — strong acne fix, opt-in because it can leak on thin/open meshes). SUN lamps with sunAngle enable PCSS and set contactHardeningLightSizeUVRatio linearly from the Blender Angle slider (0°→0, 45°→1; clamped above 45°). Loader options: { shadows?, shadowMapSize?, freezeShadows?, cleanBoneMatrixWeights? }. Exposed as level.shadowGenerators.

Static-world freeze. The scene's Freeze Shadows flag (or the freezeShadows loader option) renders each shadow map once then freezes it (REFRESHRATE_RENDER_ONCE) — a big perf win that lets you raise the map resolution, valid only when no caster moves. After moving a caster at runtime, call level.RefreshShadows() for a one-shot re-render. cleanBoneMatrixWeights scrubs bad skeleton weights on load (fixes garbled shadows on skinned meshes).

Acne defaults. Blender lamps export normalBias = 0, which leaves suns striped and point/spot edges speckled, so the loader applies a normal-bias floor when a light doesn't set its own (0.02 directional, 0.03 point/spot). It also tightens the shadow depth range automatically, since a stretched depth buffer is the usual cause of acne: directional suns get autoCalcShadowZBounds (Babylon re-fits minZ/maxZ to the casters each frame) when clip planes are left at auto; point/spot lights cap shadowMaxZ to the lamp's range (set a Custom Range in Blender for this to kick in). Explicit clip start/end or normal bias always win over these defaults.

Scene look (environment.ts, fog.ts, atmosphere.ts, postprocess.ts)

From the manifest's scene block: clear/ambient color, environment texture → IBL, fog (LINEAR/EXP/EXP2), physically based atmosphere, and post-processing are applied in await ApplySceneSettings during FinalizeLevel (environment loads asynchronously — the skybox is created only after the env texture is ready). ApplyAtmosphere runs immediately after scene settings when the manifest includes an atmosphere block. Post-processing is applied after level.Begin() so behaviors that create a runtime camera in OnStart (e.g. a script-built UniversalCamera) receive the stack on the camera that is actually active.

Environment / IBL (subsystems/environment.ts)

Image-based lighting comes from scene.environmentTexture. Without an environment block in the manifest, PBR materials get no cubemap — only direct lamps and the optional ambientColor fill. Imported glTF PBR picks up scene IBL automatically; Node Material Editor shaders need a ReflectionBlock wired to the PBRMetallicRoughness block's reflection input (leave the Reflection block's texture empty to use scene.environmentTexture). Do not wire a ReflectionTextureBlock into the Reflection block's color input unless you intend to tint — and assign it a texture, or IBL is multiplied to black.

Authoring (Babylon Scene › Environment):

A World texture always wins over Default Environment. Export copies World textures through copy_asset / save_image_asset with sanitize_asset_filename() (spaces and unsafe characters → URL-safe names under env/). The same source file always maps to the same export path; Live Link re-exports overwrite that file rather than suffixing _2, _3, … World texture discovery is shared by export and the scene UI in scene/environment.py (find_world_env_node traces World Output → Surface → Background → env/image only — orphan nodes in the node editor are ignored).

useDefault loads Babylon's built-in studio .env from the CDN at runtime — the player needs network access. Format handling at load: .env (Babylon prefiltered cube — recommended) → CubeTexture; .hdr → HDRCubeTexture; anything else → equirectangular. .exr is not loadable in-browser.

ApplyEnvironment is async: it awaits WhenTextureReady (30s timeout) before creating a skybox so createDefaultSkybox does not clone an empty cube map on the first Live Link reload.

Skybox sizing. ApplyEnvironment runs in FinalizeLevel after entities load so bounds include your level geometry. ComputeSkyboxSize() measures visible meshes (excluding skybox helper meshes) and passes max(1000, diagonal × 3) to every skybox path — the same scale Babylon's EnvironmentHelper sizeAuto uses. EnvironmentHelper skyboxes are unparented from BackgroundHelper before infiniteDistance is set (parenting disables camera-follow in Babylon). On planet-scale levels, also raise the camera Clip End in Blender — the default 1000 is often shorter than distant mesh.

Manifest example

"environment": {
  "useDefault": true,
  "intensity": 1,
  "rotationY": 0,
  "createSkybox": false
}

With skybox and fog (exports skyboxIgnoreFog when Show Skybox is on):

"environment": {
  "useDefault": true,
  "intensity": 1,
  "rotationY": 0,
  "createSkybox": true,
  "skyboxIgnoreFog": true
}

Or with an authored World texture:

"environment": {
  "file": "env/sky.env",
  "intensity": 1,
  "rotationY": 0,
  "createSkybox": false
}

Omit "environment" (or set it to null) when neither a World texture nor Default Environment is configured. skyboxIgnoreFog is only exported when createSkybox is true; older manifests without it keep Babylon's default (skybox is fogged).

Atmosphere (subsystems/atmosphere.ts)

The Babylon Atmosphere addon (@babylonjs/addons/atmosphere) provides a physically based sky and aerial perspective. It automatically integrates with PBRMaterial for consistent lighting. Requires WebGL 2 or WebGPU — the runtime calls Atmosphere.IsSupported(engine) and logs a warning if unsupported.

Authoring (Babylon Scene › Atmosphere):

When atmosphere is enabled, export forces environment.createSkybox: false — the atmosphere renders the sky. IBL from a World texture or Default Environment still loads for material reflections. For best results, enable Post-Processing › Default Pipeline (HDR) with tone mapping — the runtime sets isLinearSpaceComposition from the manifest's postProcessing.defaultPipeline.

Time of day is driven by the SUN lamp's direction in Blender (re-export after aiming the sun). The handle lives on level.atmosphere and is disposed with the level.

Manifest example

"atmosphere": {
  "pbrSunIntensity": true,
  "useLuts": true,
  "multiScatteringIntensity": 1,
  "minimumMultiScatteringIntensity": 0.1,
  "groundAlbedo": [1, 1, 1],
  "physical": {
    "peakRayleighScattering": [0.000005802, 0.000013558, 0.0000331],
    "mieScatteringScale": 1,
    "ozoneAbsorptionScale": 1,
    "originHeight": 0
  }
}

Omit "atmosphere" (or set it to null) when the panel header is off.

Render layers (subsystems/renderLayers.ts)

Babylon uses several “layer” concepts. These components map to the two used for 3D meshes:

These are not Babylon’s 2D Layer class (fullscreen background/foreground overlays) and not the logical Tag component (gameplay queries).

Authoring (Babylon Object):

Mesh ownership matches physics and reflection probes: CollectOwnedMeshes in core/meshOwnership.ts — own submeshes and primitive splits (including glTF-instanced reuse as InstancedMesh), excluding geometry that belongs to a nested child entity (a descendant node with its own bjs_id).

Applied in FinalizeLevel via ApplyRenderLayers(manifest, level) after reflection probes and before level.Begin(). Handlers only register attachment rows during the entity pass.

Manifest examples

{
  "type": "RENDERING_GROUP",
  "renderingGroupId": 2,
  "applyOwnedMeshes": true,
  "applyChildEntities": true
}

{
  "type": "LAYER_MASK",
  "layerMask": 268435456,
  "applyOwnedMeshes": true,
  "applyChildEntities": false
}

268435456 is preset Slot 0 (0x10000000). For exclusive HUD rendering, set the same mask on a second camera in application code and configure lights with includeOnlyWithLayerMask / excludeWithLayerMask as needed.

Reflection probes (subsystems/reflectionProbes.ts)

There are two reflection sources in a level:

• Scene IBL (scene.environmentTexture) — the sky / environment from Babylon Scene › Environment. Used by PBR materials everywhere by default.
• Reflection probes — local cubemaps baked from nearby meshes around a probe empty. Used only inside an authored influence volume.

A Reflection Probe component creates a Babylon ReflectionProbe, renders an authored render list into six cubemap faces, then AssignProbeMaterials sets material.reflectionTexture = probe.cubeTexture on level meshes whose bounds center falls inside the volume. Outside the volume, materials keep scene IBL.

Authoring (Babylon Object › Reflection Probe):

Probes build in FinalizeLevel after environment IBL and the single BuildNodeMaterials pass: BuildReflectionProbes → AssignProbeMaterials, before level.Begin(). Lookup: entity.GetReflectionProbe(), level.reflectionProbes.

Skybox and IBL

The visible skybox (hdrSkyBox, BackgroundSkybox, …) is a runtime mesh. AssignProbeMaterials never overrides it (IsSkyboxMesh in environment.ts) — the background sky always keeps scene IBL even when the camera is inside a probe volume.

Probes do not capture the IBL sky into their cubemap — only meshes in the render list. Glossy level surfaces inside the influence volume (water, metal, etc.) will reflect that local bake, not the HDR sky. For sky-dominated reflections, rely on Environment / IBL and keep the influence volume tight around objects that need local geometry reflections.

Limitations (v1). NME materials that wire reflections only through a ReflectionBlock (not reflectionTexture) are not auto-bound. Probes are static at load position. No probe blending — one probe per mesh. Skybox meshes cannot be added to the render list from Blender (they are not exported entities).

Manifest example

{
  "type": "REFLECTION_PROBE",
  "cubeSize": 512,
  "refreshRate": 0,
  "generateMipMaps": true,
  "renderAll": true,
  "renderList": [],
  "renderExcludes": [],
  "influenceShape": "BOX",
  "influenceSize": [4, 3, 4],
  "influenceOffset": [0, 0, 0],
  "priority": 0,
  "realTimeFiltering": true,
  "realTimeFilteringQuality": "MEDIUM"
}

Smoke test (overlapping probes). Place two empties with probes whose influence boxes overlap a glossy mesh; set probe A priority: 1 and probe B priority: 0. Export and load in the playground — the mesh should use probe A's cubemap. Set both to priority 0 to verify the closer probe wins.

Node materials (subsystems/materials/)

Standard glTF PBR from Blender rides in the .glb. For custom shaders authored in the Node Material Editor, open Properties › Material, select the material in the slot list, and use the Babylon panel (per material datablock, not per object). Export copies the JSON to materials/ and writes a top-level manifest block:

"materials": [
  {
    "name": "Water",
    "file": "materials/water.json",
    "textures": [
      { "blockId": 42, "blockName": "albedo", "file": "materials/albedo.png" }
    ],
    "inputs": [
      { "blockId": 21, "blockName": "fillColor", "type": "COLOR4", "value": [0.15, 0.4, 1, 0.35] }
    ],
    "gradients": [
      {
        "blockId": 22553,
        "blockName": "Gradient",
        "colorSteps": [
          { "step": 0, "color": { "r": 0, "g": 0, "b": 0 } },
          { "step": 1, "color": { "r": 1, "g": 1, "b": 1 } }
        ]
      }
    ]
  }
]

Matching. Runtime replaces mesh.material when mesh.material.name equals the manifest name (Blender / glTF material name). Parsed node materials are cached per manifest entry (file + name) so several Blender materials can share one NME JSON yet keep distinct texture/input/gradient overrides.

Scan NME. Click Scan NME in the Babylon material panel after picking a JSON. It lists ImageSourceBlock / TextureBlock slots, inspector-visible InputBlock parameters (visibleInInspector: true in NME — not system values like World or View), and inspector-visible GradientBlock nodes (mark the gradient block Visible in Inspector in NME). Tune floats, colors, vectors, and booleans in Parameters; edit gradient color stops (step 0–1 + RGB) in Gradients. Export copies picked images into materials/, patches texture url / name, input value, and gradient colorSteps in the exported JSON (strips embedded base64String / internalTextureLabel when overriding a texture slot). When several Blender materials share the same NME source file, export copies that JSON once and later materials patch the same exported copy in place. Particle systems use the same texture pattern on the Particles component (Scan Textures lists ParticleTextureSourceBlock slots only).

Embedded textures. NME can store image bytes inside the JSON (texture.url as a data:…;base64,… URI, or a base64String field). The runtime loads these directly from the JSON — no external file required. Embedded slots are skipped by export override logic (export does not try to replace them with empty paths). For smaller manifests and easier iteration, use Extract Textures… (bjs.extract_nme_textures): pick a folder, the add-on writes {BlockName}_{blockId}.png (or .jpg) beside the JSON, rewrites each block's url to the relative filename, then runs Scan NME and fills the matching texture rows. After that, assign or re-export like any external texture. NME saves that put the data URI only in texture.name with an empty url do not load — re-save from NME or run Extract Textures… so bytes live on texture.url.

Runtime load (two stages). Early in Load, ApplyNodeMaterials parses each NME JSON (parseSerializedObject + urlRewriter for relative URLs beside the JSON), applies manifest textures[] via BindManifestTextures (authoring wins over embedded JSON), manifest inputs[] via BindManifestInputs, and manifest gradients[] via BindManifestGradients, then assigns the result to matching meshes. Block ids from the manifest are resolved through the material's editorData.map (NME JSON id → runtime uniqueId); blockName is a fallback. Parsed materials are cached per manifest entry (file + name). No build() yet — shader compile is deferred so ReflectionBlock can see scene.environmentTexture and gradient colorSteps are baked at compile time.

Single compile. In FinalizeLevel, after await ApplySceneSettings (environment IBL when declared) and atmosphere setup, BuildNodeMaterials walks every scene NodeMaterial, awaits whenTexturesReadyAsync(), then calls build() once per unique material. Levels without NME materials or without an environment block still take this path when node materials exist — compile simply runs without scene IBL when no env texture was loaded.

Workflow. Save from NME into the level's materials/ folder (mark shader uniforms and gradient blocks Visible in Inspector when you want them in Blender; add a Reflection block for IBL) → Material Properties → Babylon panel → pick JSON → optional Extract Textures… when the JSON still embeds images → Scan NME → tune Parameters, Gradients, and assign any external texture images → export. Trace: trace-materials.html · Blender: trace-materials.

Post-processing (subsystems/postprocess.ts)

Handles land on level.post (PostProcessingHandles: pipeline?, ssao?). Use RetargetPostProcessing(handles, camera) if gameplay swaps the active camera later (default pipeline only today).

When scene.postProcessing.defaultPipeline is true, the runtime builds Babylon's DefaultRenderingPipeline (HDR on). Supported effects:

Not yet implemented (separate Babylon pipelines, not part of the default stack): SSR, TAA, motion blur, IBL shadows, frame-graph FrameGraphVolumetricLightingTask (Babylon 9's directional-light volume path; requires a frame graph render pipeline).

Manifest example

"postProcessing": {
  "defaultPipeline": true,
  "fxaa": true,
  "msaaSamples": 4,
  "bloom": { "enabled": true, "threshold": 0.9, "intensity": 0.5, "kernel": 64, "scale": 0.5 },
  "ssao": true,
  "ssaoSettings": { "radius": 2, "totalStrength": 1, "samples": 8 },
  "toneMapping": true,
  "toneMappingType": "ACES",
  "exposure": 1,
  "contrast": 1,
  "sharpen": { "enabled": false, "edgeAmount": 0.3, "colorAmount": 1 },
  "depthOfField": { "enabled": false, "blurLevel": "LOW", "focusDistance": 2000, "focalLength": 50, "fStop": 1.4 },
  "chromaticAberration": { "enabled": false, "aberrationAmount": 30 },
  "grain": { "enabled": false, "intensity": 30, "animated": false },
  "glow": { "enabled": false, "blurKernelSize": 16, "intensity": 1 },
  "vignette": { "enabled": false, "weight": 1.5 },
  "colorGrading": { "enabled": false, "file": "post/LateSunset.3dl" },
  "colorCurves": { "enabled": false, "globalHue": 30 }
}

Omit optional effect blocks when disabled; older manifests with only fxaa / bloom / toneMapping / exposure / contrast still load.

Blender UI

Babylon Scene › Environment (ui/scene_panels.py, BJS_PT_scene_environment). Default Environment (IBL without a World texture), Intensity and Rotation Y when default env is active (hidden when a World texture wins), Show Skybox (IBL-only background when off), and Skybox Ignores Fog (disabled when Show Skybox is off). The panel previews the texture on the active World Output chain via scene/environment.py.

Babylon Scene › Atmosphere (ui/scene_panels.py). Physically based sky settings on scene.bjs_scene.atmosphere (scene/atmosphere.py). Export serialization is in export/atmosphere.py.

Babylon Scene › Post-Processing (ui/post_panels.py). Settings live on scene.bjs_scene.post (scene/post_processing.py). Export serialization is in export/post_processing.py (LUT files copied via copy_asset → post/). Exposure/contrast export as 1.0 when tone mapping is off so stale panel values do not crush the image.