Skip to content

KHR_audio_emitter: channel rule, default listener, scale rule, Opus, loopCount, linear-model validation, playing pointer #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rudybear wants to merge 1 commit into
base: KHR_audio
Choose a base branch
from
+65 −13
Open

KHR_audio_emitter: channel rule, default listener, scale rule, Opus, loopCount, linear-model validation, playing pointer #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions extensions/2.0/Khronos/KHR_audio_emitter/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ The extension must be added to the file's `extensionsUsed` array and because it

Audio data objects define where audio data is located and what format the data is in. The data is either accessed via a bufferView or uri.

When storing audio data in a buffer view, the `mimeType` field must be specified. The base specification supports the `audio/mpeg` MIME type. This was chosen with consideration for the wide support for these types across 3D engines and common use cases. Other supported audio formats may be added via extensions.
When storing audio data in a buffer view, the `mimeType` field must be specified. The base specification supports the `audio/mpeg` (MP3) and `audio/ogg` (Ogg-encapsulated Opus, RFC 7845) MIME types. MP3 was chosen for its universal decode support; Opus for gapless looping and multichannel support — MP3 encoder padding makes seamless loops impossible, so loop-intended audio SHOULD use Opus. Other audio formats may be added via extensions.

Note that in tools that process glTF files, but do not implement the `KHR_audio_emitter` extension, external files referenced via the `uri` field may not be properly copied to their final destination or baked into the final binary glTF file. In these cases, using the `bufferView` property may be a better choice assuming the referenced `bufferView` index is not changed by the tool. The `uri` field might be a better choice when you want to be able to quickly change the referenced audio asset.

Expand All @@ -144,7 +144,7 @@ The `"bufferView"` property is the integer index of the bufferView that contains

#### MIME Type

The `"mimeType"` property is a string that specifies the audio's MIME type. Required if `bufferView` is defined. Unless specified by another extension, the only supported mimeType is `audio/mpeg`.
The `"mimeType"` property is a string that specifies the audio's MIME type. Required if `bufferView` is defined. Unless specified by another extension, the supported mimeTypes are `audio/mpeg` (MP3) and `audio/ogg` (Ogg Opus, RFC 7845).

#### URI

Expand All @@ -161,6 +161,7 @@ Audio sources reference audio data and define playback properties for it. Audio
| **gain** | `number` | Unitless linear multiplier against original audio file volume used for determining audio source loudness. | 1.0 |
| **playbackRate** | `number` | Multiplier for combined pitch and playback speed without resampling. | 1.0 |
| **loop** | `boolean` | Whether or not to loop the specified audio when finished. | false |
| **loopCount** | `integer` | Number of times to loop when `loop` is `true`; `0` means loop indefinitely. | 0 |
| **autoplay** | `boolean` | Whether or not to play the specified audio when the glTF is loaded. | false |
| **audio** | `number` | The index of the audio data assigned to this clip. | No audio |

Expand All @@ -180,6 +181,10 @@ For example, a value of `2.0` would double the playback speed of the audio, whic

The `"loop"` property is a boolean that specifies whether or not to loop the specified audio when finished. If `false` or not specified, the audio source does not loop.

#### Loop Count

The `"loopCount"` property is an integer that specifies how many times the audio loops when `loop` is `true`. A value of `0` (default) loops indefinitely. Ignored when `loop` is `false`.

#### Autoplay

The `"autoplay"` property is a boolean that specifies whether or not to play the specified audio when the glTF is loaded. If `false` or not specified, the audio source does not play automatically.
Expand All @@ -194,6 +199,12 @@ This value is recommended to be set to a valid index in the "audio" array, or el

Audio emitters define how audio sources are played back. Emitter properties are defined at the document level and are references by nodes. Audio may be played globally or positionally. Positional audio has further properties that define how audio volume scales with distance and angle.

**Listener**: positional audio is rendered relative to the listener. Unless overridden by another extension, the listener is the active camera (or, in applications without a glTF camera, the viewer pose): its position and orientation define the point from which spatial audio is heard.

**Channel behavior**: a positional emitter MUST down-mix its input to mono, following the Web Audio API mixing rules, prior to spatialization. A global emitter presents source channels as authored.

**Node scale**: matching `KHR_lights_punctual`, node scale does not affect emitter gain or attenuation; `refDistance` and `maxDistance` are distances in world-space units between the transformed emitter position and the listener.

#### Property Summary

| | Type | Description | Default value |
Expand Down Expand Up @@ -272,7 +283,7 @@ The `"distanceModel"` property is a string that specifies the distance model for

#### Max Distance

The `"maxDistance"` property is a number that defines the maximum distance between the emitter and listener, after which the volume will not be reduced any further. If zero or not specified, the audio emitter does not have a maximum distance, and it can be heard from any distance.
The `"maxDistance"` property is a number that defines the maximum distance between the emitter and listener, after which the volume will not be reduced any further. If zero or not specified, the audio emitter does not have a maximum distance, and it can be heard from any distance. When `distanceModel` is `"linear"`, `maxDistance` MUST be specified and MUST be greater than `refDistance` (the linear formula divides by their difference).

For the linear distance model, the max distance must be greater than the ref distance. For all distance models, max distance cannot be a negative number.

Expand Down Expand Up @@ -403,6 +414,9 @@ The following JSON pointers are defined representing mutable properties defined
| `/extensions/KHR_audio_emitter/sources/{}/gain` | `float` |
| `/extensions/KHR_audio_emitter/sources/{}/loop` | `bool` |
| `/extensions/KHR_audio_emitter/sources/{}/playbackRate` | `float` |
| `/extensions/KHR_audio_emitter/sources/{}/playing` | `bool` |

**`playing` (runtime-only pointer)**: `playing` is not a serialized property; it exists only through the Object Model, enabling event-driven playback from `KHR_interactivity` and `KHR_animation_pointer` without dedicated audio nodes. Writing `true` (re)starts playback of the source on every emitter that references it — a rising edge always restarts from the beginning (`offset` semantics, where defined by other extensions, apply). Writing `false` stops all playing instances of the source. Reading returns `true` while at least one instance is playing. This makes interactive assets (e.g., a playable instrument) expressible today; see the working demonstration linked from KhronosGroup/glTF#2561.

Additionally, the following JSON pointers are defined for read-only properties:

Expand Down
36 changes: 30 additions & 6 deletions extensions/2.0/Khronos/KHR_audio_emitter/schema/glTF.KHR_audio_emitter.audio.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
"title": "KHR_audio_emitter Audio Data",
"type": "object",
"description": "Audio data objects define where audio data is located and what format the data is in. The data is either accessed via a bufferView or uri. `mimeType` is required when `bufferView` is used.",
"allOf": [{ "$ref": "glTFChildOfRootProperty.schema.json" }],
"allOf": [
{
"$ref": "glTFChildOfRootProperty.schema.json"
}
],
"properties": {
"uri": {
"type": "string",
Expand All @@ -15,26 +19,46 @@
},
"mimeType": {
"type": "string",
"description": "The audio's MIME type. Required if `bufferView` is defined. Unless specified by another extension, the only supported mimeType is `audio/mpeg`.",
"description": "The audio MIME type. Required if bufferView is defined. Unless specified by another extension, the supported mimeTypes are audio/mpeg (MP3) and audio/ogg (Ogg Opus, RFC 7845).",
"anyOf": [
{
"const": "audio/mpeg"
},
{
"const": "audio/ogg"
},
{
"type": "string"
}
]
},
"bufferView": {
"allOf": [{ "$ref": "glTFid.schema.json" }],
"allOf": [
{
"$ref": "glTFid.schema.json"
}
],
"description": "The index of the bufferView that contains the audio data. Use this instead of the audio source's uri property."
},
"name": {},
"extensions": {},
"extras": {}
},
"dependentRequired": {
"bufferView": ["mimeType"]
"bufferView": [
"mimeType"
]
},
"oneOf": [{ "required": ["uri"] }, { "required": ["bufferView"] }]
}
"oneOf": [
{
"required": [
"uri"
]
},
{
"required": [
"bufferView"
]
}
]
}
22 changes: 18 additions & 4 deletions extensions/2.0/Khronos/KHR_audio_emitter/schema/glTF.KHR_audio_emitter.source.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
"title": "KHR_audio_emitter Audio Source",
"type": "object",
"description": "Audio sources reference audio data and define playback properties for it. Audio sources may be used by zero to many audio emitters.",
"allOf": [{ "$ref": "glTFChildOfRootProperty.schema.json" }],
"allOf": [
{
"$ref": "glTFChildOfRootProperty.schema.json"
}
],
"properties": {
"autoplay": {
"type": "boolean",
Expand All @@ -28,11 +32,21 @@
"default": false
},
"audio": {
"allOf": [ { "$ref": "glTFid.schema.json" } ],
"allOf": [
{
"$ref": "glTFid.schema.json"
}
],
"description": "The index of the audio data assigned to this clip."
},
"name": {},
"extensions": {},
"extras": {}
"extras": {},
"loopCount": {
"type": "integer",
"minimum": 0,
"default": 0,
"description": "Number of times to loop when loop is true; 0 means loop indefinitely. Ignored when loop is false."
}
}
}
}