Table of Contents

Class LiveMusicClient

Namespace
GenerativeAI.Live
Assembly
GenerativeAI.Live.dll

[Experimental] A client for real-time music generation with Lyria RealTime over the Gemini API Live Music WebSocket endpoint (BidiGenerateMusic).

public class LiveMusicClient : IDisposable
Inheritance
LiveMusicClient
Implements
Inherited Members
Extension Methods

Remarks

Typical usage: ConnectAsync(bool, CancellationToken), then SetWeightedPromptsAsync(IEnumerable<WeightedPrompt>, CancellationToken) and optionally SetMusicGenerationConfigAsync(LiveMusicGenerationConfig, CancellationToken), then PlayAsync(CancellationToken). Audio arrives through AudioChunkReceived as raw 16-bit PCM (48 kHz stereo by default). Prompts and config can be changed while playing to steer the music. Live Music is only available on the Gemini API (not Vertex AI).

Constructors

LiveMusicClient(IPlatformAdapter, string, ILogger?)

Initializes a new instance of the LiveMusicClient class.

public LiveMusicClient(IPlatformAdapter platformAdapter, string modelName = "models/lyria-realtime-exp", ILogger? logger = null)

Parameters

platformAdapter IPlatformAdapter

The Gemini API platform adapter (a GoogleAIPlatformAdapter) that provides the API key.

modelName string

The model to use. Defaults to LyriaRealtimeExp.

logger ILogger

An optional logger.

Exceptions

NotSupportedException

The platform adapter is not a Gemini API adapter (Live Music is not supported on Vertex AI).

LiveMusicClient(string?, string, ILogger?)

Initializes a new instance of the LiveMusicClient class with a Gemini API key.

public LiveMusicClient(string? apiKey, string modelName = "models/lyria-realtime-exp", ILogger? logger = null)

Parameters

apiKey string

The Gemini API key. When null, the GOOGLE_API_KEY environment variable is used.

modelName string

The model to use. Defaults to LyriaRealtimeExp.

logger ILogger

An optional logger.

Fields

DefaultChannels

The default number of channels of Lyria RealTime audio output.

public const int DefaultChannels = 2

Field Value

int

DefaultSampleRate

The default sample rate of Lyria RealTime audio output, in Hz.

public const int DefaultSampleRate = 48000

Field Value

int

Properties

ApiVersion

Gets or sets the API version used in the WebSocket URL. Defaults to v1alpha.

public string ApiVersion { get; set; }

Property Value

string

Client

Gets the WebSocket client used for the current connection, or null when not connected.

public IWebsocketClient? Client { get; }

Property Value

IWebsocketClient

IsSetupComplete

Gets whether the server has acknowledged the setup message on the current connection.

public bool IsSetupComplete { get; }

Property Value

bool

ModelName

Gets the fully qualified model name sent in the setup message (for example models/lyria-realtime-exp).

public string ModelName { get; }

Property Value

string

RestoreStateOnReconnect

Gets or sets whether, after the underlying WebSocket reconnects, the client re-sends the setup message and then restores the last weighted prompts, generation config and (if playing) the PLAY signal. Defaults to true.

public bool RestoreStateOnReconnect { get; set; }

Property Value

bool

SetupTimeout

Gets or sets how long ConnectAsync(bool, CancellationToken) waits for the server's setupComplete message. Defaults to 30 seconds.

public TimeSpan SetupTimeout { get; set; }

Property Value

TimeSpan

WebsocketClientFactory

Optional factory that ConnectAsync(bool, CancellationToken) uses to create the WebSocket client instead of the default reconnecting Websocket.Client.WebsocketClient. It receives the Live Music endpoint URL (which already contains the API key as the key query parameter) and must return a client that has not been started; ConnectAsync(bool, CancellationToken) subscribes to it and then calls Websocket.Client.IWebsocketClient.Start(). Useful for custom transports, proxies and unit testing.

public Func<string, IWebsocketClient>? WebsocketClientFactory { get; set; }

Property Value

Func<string, IWebsocketClient>

Methods

ConnectAsync(bool, CancellationToken)

Connects to the Live Music endpoint and sends the setup message with the model name.

public Task ConnectAsync(bool waitForSetupComplete = true, CancellationToken cancellationToken = default)

Parameters

waitForSetupComplete bool

When true (the default), waits until the server acknowledges the setup with setupComplete (or SetupTimeout elapses) before returning, as the protocol requires before sending other messages.

cancellationToken CancellationToken

A token to cancel waiting for the setup acknowledgement.

Returns

Task

A task representing the asynchronous operation.

Exceptions

TimeoutException

The server did not acknowledge the setup within SetupTimeout.

DisconnectAsync(CancellationToken)

Closes the WebSocket connection and releases its resources.

public Task DisconnectAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

Dispose()

Disconnects and releases the resources used by the LiveMusicClient.

public void Dispose()

Dispose(bool)

Releases the resources used by the LiveMusicClient.

protected virtual void Dispose(bool disposing)

Parameters

disposing bool

true to release managed resources.

GetLiveMusicUrl()

Builds the Live Music WebSocket URL, including the API key.

public string GetLiveMusicUrl()

Returns

string

The WebSocket URL.

PauseAsync(CancellationToken)

Pauses music generation ({"playbackControl":"PAUSE"}). Use PlayAsync(CancellationToken) to resume.

public Task PauseAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

PlayAsync(CancellationToken)

Starts (or resumes) music generation ({"playbackControl":"PLAY"}).

public Task PlayAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

ResetContextAsync(CancellationToken)

Resets the generation context without stopping the music; prompts and config are retained ({"playbackControl":"RESET_CONTEXT"}).

public Task ResetContextAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

SendAsync(LiveMusicClientMessage, CancellationToken)

Sends a raw client message over the WebSocket.

public Task SendAsync(LiveMusicClientMessage message, CancellationToken cancellationToken = default)

Parameters

message LiveMusicClientMessage

The message to send.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

Exceptions

InvalidOperationException

The client is not connected.

SendPlaybackControlAsync(LiveMusicPlaybackControl, CancellationToken)

Sends a playback control signal ({"playbackControl":"..."}).

public Task SendPlaybackControlAsync(LiveMusicPlaybackControl playbackControl, CancellationToken cancellationToken = default)

Parameters

playbackControl LiveMusicPlaybackControl

The playback control signal.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

SendSetupAsync(CancellationToken)

Sends the setup message ({"setup":{"model":"models/..."}}). Called by ConnectAsync(bool, CancellationToken).

public Task SendSetupAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

SetMusicGenerationConfigAsync(LiveMusicGenerationConfig, CancellationToken)

Sets the music generation config ({"musicGenerationConfig":{...}}). Changes to bpm or scale take full effect after ResetContextAsync(CancellationToken) (or stop/play).

public Task SetMusicGenerationConfigAsync(LiveMusicGenerationConfig config, CancellationToken cancellationToken = default)

Parameters

config LiveMusicGenerationConfig

The music generation config.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

SetWeightedPromptsAsync(IEnumerable<WeightedPrompt>, CancellationToken)

Sets the weighted prompts that steer the music ({"clientContent":{"weightedPrompts":[...]}}). Weights must not be 0; the server normalizes the weights of all prompts in the message.

public Task SetWeightedPromptsAsync(IEnumerable<WeightedPrompt> prompts, CancellationToken cancellationToken = default)

Parameters

prompts IEnumerable<WeightedPrompt>

The weighted prompts.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

SetWeightedPromptsAsync(string, CancellationToken)

Sets a single text prompt with weight 1.0.

public Task SetWeightedPromptsAsync(string text, CancellationToken cancellationToken = default)

Parameters

text string

The text prompt.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

StopAsync(CancellationToken)

Stops music generation and resets the context while retaining prompts and config ({"playbackControl":"STOP"}). Use PlayAsync(CancellationToken) to restart.

public Task StopAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task

A task representing the asynchronous operation.

WaitForSetupCompleteAsync(CancellationToken)

Waits until the server acknowledges the setup message on the current connection.

public Task WaitForSetupCompleteAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

A token to cancel the wait.

Returns

Task

A task that completes when setupComplete has been received.

Exceptions

TimeoutException

The server did not acknowledge the setup within SetupTimeout.

Events

AudioChunkReceived

Occurs when the server sends an audio chunk. Raised once per chunk, in order.

public event EventHandler<LiveMusicAudioChunkReceivedEventArgs>? AudioChunkReceived

Event Type

EventHandler<LiveMusicAudioChunkReceivedEventArgs>

ClientCreated

Occurs after the WebSocket client is created, before the connection is started.

public event EventHandler<ClientCreatedEventArgs>? ClientCreated

Event Type

EventHandler<ClientCreatedEventArgs>

Connected

Occurs when the WebSocket connection is established (before setup is acknowledged).

public event EventHandler? Connected

Event Type

EventHandler

Disconnected

Occurs when the WebSocket connection is closed. When the server sent a close reason, the event args are an ErrorMessageEventArgs carrying it.

public event EventHandler? Disconnected

Event Type

EventHandler

ErrorOccurred

Occurs when an error happens while connecting, sending or processing messages (including exceptions thrown by event handlers).

public event EventHandler<ErrorEventArgs>? ErrorOccurred

Event Type

EventHandler<ErrorEventArgs>

FilteredPromptReceived

Occurs when the server reports that a prompt was filtered.

public event EventHandler<LiveMusicFilteredPromptEventArgs>? FilteredPromptReceived

Event Type

EventHandler<LiveMusicFilteredPromptEventArgs>

MessageReceived

Occurs for every message received from the server, after the specific events have been raised.

public event EventHandler<LiveMusicMessageReceivedEventArgs>? MessageReceived

Event Type

EventHandler<LiveMusicMessageReceivedEventArgs>

SetupCompleted

Occurs when the server acknowledges the setup message (setupComplete).

public event EventHandler? SetupCompleted

Event Type

EventHandler

See Also