Class LiveMusicClient
- Namespace
- GenerativeAI.Live
- Assembly
- GenerativeAI.Live.dll
[Experimental] A client for real-time music generation with Lyria RealTime over the Gemini API Live Music
WebSocket endpoint (BidiGenerateMusic).
public class LiveMusicClient : IDisposable
- Inheritance
-
LiveMusicClient
- Implements
- Inherited Members
- Extension Methods
Remarks
Typical usage: ConnectAsync(bool, CancellationToken), then SetWeightedPromptsAsync(IEnumerable<WeightedPrompt>, CancellationToken) and optionally SetMusicGenerationConfigAsync(LiveMusicGenerationConfig, CancellationToken), then PlayAsync(CancellationToken). Audio arrives through AudioChunkReceived as raw 16-bit PCM (48 kHz stereo by default). Prompts and config can be changed while playing to steer the music. Live Music is only available on the Gemini API (not Vertex AI).
Constructors
LiveMusicClient(IPlatformAdapter, string, ILogger?)
Initializes a new instance of the LiveMusicClient class.
public LiveMusicClient(IPlatformAdapter platformAdapter, string modelName = "models/lyria-realtime-exp", ILogger? logger = null)
Parameters
platformAdapterIPlatformAdapterThe Gemini API platform adapter (a GoogleAIPlatformAdapter) that provides the API key.
modelNamestringThe model to use. Defaults to LyriaRealtimeExp.
loggerILoggerAn optional logger.
Exceptions
- NotSupportedException
The platform adapter is not a Gemini API adapter (Live Music is not supported on Vertex AI).
LiveMusicClient(string?, string, ILogger?)
Initializes a new instance of the LiveMusicClient class with a Gemini API key.
public LiveMusicClient(string? apiKey, string modelName = "models/lyria-realtime-exp", ILogger? logger = null)
Parameters
apiKeystringThe Gemini API key. When null, the
GOOGLE_API_KEYenvironment variable is used.modelNamestringThe model to use. Defaults to LyriaRealtimeExp.
loggerILoggerAn optional logger.
Fields
DefaultChannels
The default number of channels of Lyria RealTime audio output.
public const int DefaultChannels = 2
Field Value
DefaultSampleRate
The default sample rate of Lyria RealTime audio output, in Hz.
public const int DefaultSampleRate = 48000
Field Value
Properties
ApiVersion
Gets or sets the API version used in the WebSocket URL. Defaults to v1alpha.
public string ApiVersion { get; set; }
Property Value
Client
Gets the WebSocket client used for the current connection, or null when not connected.
public IWebsocketClient? Client { get; }
Property Value
- IWebsocketClient
IsSetupComplete
Gets whether the server has acknowledged the setup message on the current connection.
public bool IsSetupComplete { get; }
Property Value
ModelName
Gets the fully qualified model name sent in the setup message (for example models/lyria-realtime-exp).
public string ModelName { get; }
Property Value
RestoreStateOnReconnect
Gets or sets whether, after the underlying WebSocket reconnects, the client re-sends the setup message and then
restores the last weighted prompts, generation config and (if playing) the PLAY signal. Defaults to true.
public bool RestoreStateOnReconnect { get; set; }
Property Value
SetupTimeout
Gets or sets how long ConnectAsync(bool, CancellationToken) waits for the server's setupComplete message.
Defaults to 30 seconds.
public TimeSpan SetupTimeout { get; set; }
Property Value
WebsocketClientFactory
Optional factory that ConnectAsync(bool, CancellationToken) uses to create the WebSocket client instead of the default
reconnecting Websocket.Client.WebsocketClient. It receives the Live Music endpoint URL (which already contains the
API key as the key query parameter) and must return a client that has not been started;
ConnectAsync(bool, CancellationToken) subscribes to it and then calls Websocket.Client.IWebsocketClient.Start().
Useful for custom transports, proxies and unit testing.
public Func<string, IWebsocketClient>? WebsocketClientFactory { get; set; }
Property Value
Methods
ConnectAsync(bool, CancellationToken)
Connects to the Live Music endpoint and sends the setup message with the model name.
public Task ConnectAsync(bool waitForSetupComplete = true, CancellationToken cancellationToken = default)
Parameters
waitForSetupCompleteboolWhen true (the default), waits until the server acknowledges the setup with
setupComplete(or SetupTimeout elapses) before returning, as the protocol requires before sending other messages.cancellationTokenCancellationTokenA token to cancel waiting for the setup acknowledgement.
Returns
- Task
A task representing the asynchronous operation.
Exceptions
- TimeoutException
The server did not acknowledge the setup within SetupTimeout.
DisconnectAsync(CancellationToken)
Closes the WebSocket connection and releases its resources.
public Task DisconnectAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
Dispose()
Disconnects and releases the resources used by the LiveMusicClient.
public void Dispose()
Dispose(bool)
Releases the resources used by the LiveMusicClient.
protected virtual void Dispose(bool disposing)
Parameters
disposingbooltrue to release managed resources.
GetLiveMusicUrl()
Builds the Live Music WebSocket URL, including the API key.
public string GetLiveMusicUrl()
Returns
- string
The WebSocket URL.
PauseAsync(CancellationToken)
Pauses music generation ({"playbackControl":"PAUSE"}). Use PlayAsync(CancellationToken) to resume.
public Task PauseAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
PlayAsync(CancellationToken)
Starts (or resumes) music generation ({"playbackControl":"PLAY"}).
public Task PlayAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
ResetContextAsync(CancellationToken)
Resets the generation context without stopping the music; prompts and config are retained
({"playbackControl":"RESET_CONTEXT"}).
public Task ResetContextAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
SendAsync(LiveMusicClientMessage, CancellationToken)
Sends a raw client message over the WebSocket.
public Task SendAsync(LiveMusicClientMessage message, CancellationToken cancellationToken = default)
Parameters
messageLiveMusicClientMessageThe message to send.
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
Exceptions
- InvalidOperationException
The client is not connected.
SendPlaybackControlAsync(LiveMusicPlaybackControl, CancellationToken)
Sends a playback control signal ({"playbackControl":"..."}).
public Task SendPlaybackControlAsync(LiveMusicPlaybackControl playbackControl, CancellationToken cancellationToken = default)
Parameters
playbackControlLiveMusicPlaybackControlThe playback control signal.
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
SendSetupAsync(CancellationToken)
Sends the setup message ({"setup":{"model":"models/..."}}). Called by ConnectAsync(bool, CancellationToken).
public Task SendSetupAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
SetMusicGenerationConfigAsync(LiveMusicGenerationConfig, CancellationToken)
Sets the music generation config ({"musicGenerationConfig":{...}}). Changes to bpm or
scale take full effect after ResetContextAsync(CancellationToken) (or stop/play).
public Task SetMusicGenerationConfigAsync(LiveMusicGenerationConfig config, CancellationToken cancellationToken = default)
Parameters
configLiveMusicGenerationConfigThe music generation config.
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
SetWeightedPromptsAsync(IEnumerable<WeightedPrompt>, CancellationToken)
Sets the weighted prompts that steer the music ({"clientContent":{"weightedPrompts":[...]}}).
Weights must not be 0; the server normalizes the weights of all prompts in the message.
public Task SetWeightedPromptsAsync(IEnumerable<WeightedPrompt> prompts, CancellationToken cancellationToken = default)
Parameters
promptsIEnumerable<WeightedPrompt>The weighted prompts.
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
SetWeightedPromptsAsync(string, CancellationToken)
Sets a single text prompt with weight 1.0.
public Task SetWeightedPromptsAsync(string text, CancellationToken cancellationToken = default)
Parameters
textstringThe text prompt.
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
StopAsync(CancellationToken)
Stops music generation and resets the context while retaining prompts and config
({"playbackControl":"STOP"}). Use PlayAsync(CancellationToken) to restart.
public Task StopAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the operation.
Returns
- Task
A task representing the asynchronous operation.
WaitForSetupCompleteAsync(CancellationToken)
Waits until the server acknowledges the setup message on the current connection.
public Task WaitForSetupCompleteAsync(CancellationToken cancellationToken = default)
Parameters
cancellationTokenCancellationTokenA token to cancel the wait.
Returns
- Task
A task that completes when
setupCompletehas been received.
Exceptions
- TimeoutException
The server did not acknowledge the setup within SetupTimeout.
Events
AudioChunkReceived
Occurs when the server sends an audio chunk. Raised once per chunk, in order.
public event EventHandler<LiveMusicAudioChunkReceivedEventArgs>? AudioChunkReceived
Event Type
ClientCreated
Occurs after the WebSocket client is created, before the connection is started.
public event EventHandler<ClientCreatedEventArgs>? ClientCreated
Event Type
Connected
Occurs when the WebSocket connection is established (before setup is acknowledged).
public event EventHandler? Connected
Event Type
Disconnected
Occurs when the WebSocket connection is closed. When the server sent a close reason, the event args are an ErrorMessageEventArgs carrying it.
public event EventHandler? Disconnected
Event Type
ErrorOccurred
Occurs when an error happens while connecting, sending or processing messages (including exceptions thrown by event handlers).
public event EventHandler<ErrorEventArgs>? ErrorOccurred
Event Type
FilteredPromptReceived
Occurs when the server reports that a prompt was filtered.
public event EventHandler<LiveMusicFilteredPromptEventArgs>? FilteredPromptReceived
Event Type
MessageReceived
Occurs for every message received from the server, after the specific events have been raised.
public event EventHandler<LiveMusicMessageReceivedEventArgs>? MessageReceived
Event Type
SetupCompleted
Occurs when the server acknowledges the setup message (setupComplete).
public event EventHandler? SetupCompleted