Table of Contents

Class LiveClientRealtimeInput

Namespace
GenerativeAI.Types
Assembly
GenerativeAI.dll

User input that is sent in real time. This is different from LiveClientContent in a few ways:

  • Can be sent continuously without interruption to model generation.
  • If there is a need to mix data interleaved across the LiveClientContent and the LiveClientRealtimeInput, the server attempts to optimize for best response, but there are no guarantees.
  • End of turn is not explicitly specified, but is rather derived from user activity (for example, end of speech).
  • Even before the end of turn, the data is processed incrementally to optimize for a fast start of the response from the model.
  • Is always direct user input that is sent in real time. Can be sent continuously without interruptions. The model automatically detects the beginning and the end of user speech and starts or terminates streaming the response accordingly. Data is processed incrementally as it arrives, minimizing latency.
public class LiveClientRealtimeInput
Inheritance
LiveClientRealtimeInput
Inherited Members
Extension Methods

Properties

ActivityEnd

Marks the end of user activity. This can only be sent if automatic (i.e. server-side) activity detection is disabled.

[JsonPropertyName("activityEnd")]
public ActivityEnd? ActivityEnd { get; set; }

Property Value

ActivityEnd

ActivityStart

Marks the start of user activity. This can only be sent if automatic (i.e. server-side) activity detection is disabled.

[JsonPropertyName("activityStart")]
public ActivityStart? ActivityStart { get; set; }

Property Value

ActivityStart

Audio

Inline audio data sent in real time.

[JsonPropertyName("audio")]
public Blob? Audio { get; set; }

Property Value

Blob

AudioStreamEnd

Indicates that the audio stream has ended, e.g. because the microphone was turned off. This should only be sent when automatic activity detection is enabled (which is the default). The client can reopen the stream by sending an audio message. Gemini Developer API only; not supported by Vertex AI.

[JsonPropertyName("audioStreamEnd")]
public bool? AudioStreamEnd { get; set; }

Property Value

bool?

MediaChunks

Inlined bytes data for media input.

[Obsolete("mediaChunks is deprecated by the Gemini API. Use Audio, Video, or Text instead.")]
[JsonPropertyName("mediaChunks")]
public Blob[]? MediaChunks { get; set; }

Property Value

Blob[]

Remarks

Deprecated by the Gemini API. Use Audio, Video, or Text instead.

Text

Text input sent in real time.

[JsonPropertyName("text")]
public string? Text { get; set; }

Property Value

string

Video

Inline video data sent in real time.

[JsonPropertyName("video")]
public Blob? Video { get; set; }

Property Value

Blob

See Also