In this post we will go over the modules for deserializing and serializing the audio samples in our ZedBoard Audio Processor using the I2S protocol.
In the previous post we defined the top level of our Audio Processor Core, where all the audio processing and analysis in our design will take place. We used a passthrough as a quick and dirty way to get the audio out of the ZedBoard, and established the need for serializing/deserializing the data to do something meaningful with it. We’ll now take a detailed look at how the (de)serialization works.
Audio Data Format and Transmission Procotol
As mentioned in the previous post, we configured the audio codec on the ZedBoard to transmit and receive the audio data using the I2S protocol. As a quick reminder, the figure below shows the timing diagram for I2S.
For more details regarding the data format and protocol timing you can go back to the previous post. Let’s jump right in with the deserializer logic.
Audio Deserializer
Before we can start deserializing the data, we need to keep in mind that the signals generated by the audio codec are asynchronous to the clock domain that we use to drive the logic in the FPGA. Therefore, we need to use a synchronizer circuit to minimize the risk of metastability at the clock domain crossing. We use a double register synchronizer and add the suffixes ‘meta‘ and ‘stable‘ to each of the register stages to make it easier to keep track of them.
After the signals from the audio codec have been synchronized, we need to add edge detection to the left/right and bit clocks. We do so by adding a third (delay) register to the synchronized left/right and bit clock signals and comparing the values of the last two registers.
- If the second (stable) register is high, and the third (delay) register is low, a rising edge has occurred
- If the second (stable) register is low, and the third (delay) register is high, a falling edge has occurred
Once the cock-domain crossing and edge detection are in place, we are ready to start deserializing the data. The deserialization process goes as follows:
- Wait for a falling edge in the left/right clock signal
- Wait for the next falling edge of the bit clock (accounts for the one-cycle delay in the I2S protocol before the first bit of audio data is shifted)
- For the next twenty-four rising edges of the bit clock capture the left audio sample by storing it in a shift register. The shift register then contains the deserialized sample of the left channel
- After the 24th bit of the left channel has been stored, wait for the rising edge of the left/right clock signal
- Wait for the next falling edge of the bit clock (accounts for the one-cycle delay in the I2S protocol before the first bit of audio data is shifted)
- For the next twenty-four rising edges of the bit clock capture the right audio sample by storing it in a shift register. The shift register then contains the deserialized sample of the right channel
- After the 24th bit of the right channel has been stored, assign the content of both shift registers to their corresponding outputs, and enable the ‘o_data_valid‘ output signal to indicate that the deserialized data can be read by the downstream logic.
The deserializing logic is implemented in an FSM. The code below shows the complete RTL description for the Audio Deserializer, including the cock-domain crossing and edge detection, the main FSM and the IO and signal declarations.
module audio_deserializer (
input logic i_clock,
// I2S Interface
input logic i_codec_bit_clock,
input logic i_codec_lr_clock,
input logic i_codec_adc_data,
// Parallel Data Output
output logic [23 : 0] o_data_left,
output logic [23 : 0] o_data_right,
output logic o_data_valid
);
timeunit 1ns;
timeprecision 1ps;
logic codec_adc_data_meta;
logic codec_adc_data_stable;
logic codec_bit_clock_meta;
logic codec_bit_clock_stable;
logic codec_bit_clock_delay;
logic codec_bit_clock_rising;
logic codec_bit_clock_falling;
logic codec_lr_clock_meta;
logic codec_lr_clock_stable;
logic codec_lr_clock_delay;
logic codec_lr_clock_rising;
logic codec_lr_clock_falling;
logic [4 : 0] bit_counter;
logic signed [23 : 0] shift_register_left;
logic signed [23 : 0] shift_register_right;
logic data_valid;
// Edge detection for the bit and lr clocks
always_ff @(posedge i_clock) begin
// Synchronize the audio signals to i_clock, delay the codec lr and bit clock signals
codec_adc_data_meta <= i_codec_adc_data;
codec_adc_data_stable <= codec_adc_data_meta;
codec_bit_clock_meta <= i_codec_bit_clock;
codec_bit_clock_stable <= codec_bit_clock_meta;
codec_bit_clock_delay <= codec_bit_clock_stable;
codec_lr_clock_meta <= i_codec_lr_clock;
codec_lr_clock_stable <= codec_lr_clock_meta;
codec_lr_clock_delay <= codec_lr_clock_stable;
// Detect bit clock rising/falling
if ((codec_bit_clock_stable == 1\'b1) & (codec_bit_clock_delay == 1\'b0)) begin
codec_bit_clock_rising <= 1\'b1;
end else begin
codec_bit_clock_rising <= 1\'b0;
end
if ((codec_bit_clock_stable == 1\'b0) & (codec_bit_clock_delay == 1\'b1)) begin
codec_bit_clock_falling <= 1\'b1;
end else begin
codec_bit_clock_falling <= 1\'b0;
end
// Detect lr clock rising/falling
if ((codec_lr_clock_stable == 1\'b1) & (codec_lr_clock_delay == 1\'b0)) begin
codec_lr_clock_rising <= 1\'b1;
end else begin
codec_lr_clock_rising <= 1\'b0;
end
if ((codec_lr_clock_stable == 1\'b0) & (codec_lr_clock_delay == 1\'b1)) begin
codec_lr_clock_falling <= 1\'b1;
end else begin
codec_lr_clock_falling <= 1\'b0;
end
end
// Main FSM
enum logic [2 : 0] {IDLE,
LR_CLOCK_FALLING,
LEFT_DATA_SHIFT,
WAIT_LR_CLOCK_RISING,
LR_CLOCK_RISING,
RIGHT_DATA_SHIFT,
OUTPUT_GEN} fsm_state = IDLE;
always_ff @(posedge i_clock) begin
case (fsm_state)
IDLE : begin
bit_counter <= \'b0;
shift_register_left <= \'b0;
shift_register_right <= \'b0;
o_data_left <= \'b0;
o_data_right <= \'b0;
o_data_valid <= 1\'b0;
data_valid <= 1\'b0;
if (codec_lr_clock_falling == 1\'b1) begin
fsm_state <= LR_CLOCK_FALLING;
end
end
LR_CLOCK_FALLING : begin
if (codec_bit_clock_rising == 1\'b1) begin
fsm_state <= LEFT_DATA_SHIFT;
end
end
LEFT_DATA_SHIFT : begin
if (codec_bit_clock_rising == 1\'b1) begin
bit_counter <= bit_counter + 1;
shift_register_left = {shift_register_left[22:0], codec_adc_data_stable};
end
if (bit_counter == 24) begin
bit_counter <= \'b0;
fsm_state <= WAIT_LR_CLOCK_RISING;
end
end
WAIT_LR_CLOCK_RISING : begin
if (codec_lr_clock_rising == 1\'b1) begin
fsm_state <= LR_CLOCK_RISING;
end
end
LR_CLOCK_RISING : begin
if (codec_bit_clock_rising == 1\'b1) begin
fsm_state <= RIGHT_DATA_SHIFT;
end
end
RIGHT_DATA_SHIFT : begin
if (codec_bit_clock_rising == 1\'b1) begin
bit_counter <= bit_counter + 1;
shift_register_right = {shift_register_right[22:0], codec_adc_data_stable};
end
if (bit_counter == 24) begin
bit_counter <= \'b0;
fsm_state <= OUTPUT_GEN;
end
end
OUTPUT_GEN : begin
o_data_left <= shift_register_left;
o_data_right <= shift_register_right;
o_data_valid <= 1\'b1;
data_valid <= 1\'b1;
fsm_state <= IDLE;
end
default : begin
fsm_state <= IDLE;
o_data_left <= \'b0;
o_data_right <= \'b0;
o_data_valid <= 1\'b0;
data_valid <= 1\'b0;
end
endcase
end
endmoduleAudio Serializer
Now that we have deserialized our audio data, we can process and analyze it within the FPGA. However, eventually we will need to send it back to the audio coded so that it can covert it back to an analog signal. The job of the Audio Serializer is to convert the deserialized data back to a serial stream that we can send to the audio coded over I2S.
The same considerations regarding signal synchronization and metastability of the I2S input signals that we discussed for the Audio Deserializer apply for the Audio Serializer. However, this time only the left/right and bit clock signals need to be synchronized, since we are the ones generating the data signal. The audio codec is responsible for appropriately synchronizing the data signal into its clock domain.
The Audio Serializer implements the edge detection logic for the left/right and bit clocks in the exact same way as the Audio Deserializer.
Once the cock-domain crossing and edge detection are in place, we are ready to start serializing the data. The serialization process goes as follows:
- Wait for the ‘i_data_valid‘ to be asserted, which indicates that an audio sample is ready to be serialized.
- Wait for a falling edge in the left/right clock signal
- Wait for the next falling edge of the bit clock (accounts for the one-cycle delay in the I2S protocol before the first bit of audio data is shifted)
- For the next twenty-four falling edges of the bit clock shift out each bit of the left audio sample, starting with the most significant bit
- After the least significant bit of the left channel has been shifted, wait for the rising edge of the left/right clock signal
- Wait for the next rising edge of the bit clock (accounts for the one-cycle delay in the I2S protocol before the first bit of audio data is shifted)
- For the next twenty-four falling edges of the bit clock shift out each bit of the left audio sample, starting with the most significant bit
The serializing logic is implemented in an FSM. The code below shows the complete RTL description for the Audio Serializer, including the cock-domain crossing and edge detection, the main FSM and the IO and signal declarations.
module audio_serializer (
input logic i_clock,
// I2S Interface
input logic i_codec_bit_clock,
input logic i_codec_lr_clock,
output logic o_codec_dac_data,
// Parallel Data Input
input logic [23 : 0] i_data_left,
input logic [23 : 0] i_data_right,
input logic i_data_valid
);
timeunit 1ns;
timeprecision 1ps;
logic codec_bit_clock_meta;
logic codec_bit_clock_stable;
logic codec_bit_clock_delay;
logic codec_bit_clock_rising;
logic codec_bit_clock_falling;
logic codec_lr_clock_meta;
logic codec_lr_clock_stable;
logic codec_lr_clock_delay;
logic codec_lr_clock_rising;
logic codec_lr_clock_falling;
logic [4 : 0] bit_counter;
logic signed [23 : 0] shift_register_left;
logic signed [23 : 0] shift_register_right;
// Edge detection for the bit and lr clocks
always_ff @(posedge i_clock) begin
// Synchronize the audio signals to i_clock, delay the codec lr and bit clock signals
codec_bit_clock_meta <= i_codec_bit_clock;
codec_bit_clock_stable <= codec_bit_clock_meta;
codec_bit_clock_delay <= codec_bit_clock_stable;
codec_lr_clock_meta <= i_codec_lr_clock;
codec_lr_clock_stable <= codec_lr_clock_meta;
codec_lr_clock_delay <= codec_lr_clock_stable;
// Detect bit clock rising/falling
if ((codec_bit_clock_stable == 1\'b1) & (codec_bit_clock_delay == 1\'b0)) begin
codec_bit_clock_rising <= 1\'b1;
end else begin
codec_bit_clock_rising <= 1\'b0;
end
if ((codec_bit_clock_stable == 1\'b0) & (codec_bit_clock_delay == 1\'b1)) begin
codec_bit_clock_falling <= 1\'b1;
end else begin
codec_bit_clock_falling <= 1\'b0;
end
// Detect lr clock rising/falling
if ((codec_lr_clock_stable == 1\'b1) & (codec_lr_clock_delay == 1\'b0)) begin
codec_lr_clock_rising <= 1\'b1;
end else begin
codec_lr_clock_rising <= 1\'b0;
end
if ((codec_lr_clock_stable == 1\'b0) & (codec_lr_clock_delay == 1\'b1)) begin
codec_lr_clock_falling <= 1\'b1;
end else begin
codec_lr_clock_falling <= 1\'b0;
end
end
// Main FSM
enum logic [2 : 0] {IDLE,
WAIT_LR_CLOCK_FALLING,
LR_CLOCK_FALLING,
LEFT_DATA_SHIFT,
WAIT_LR_CLOCK_RISING,
LR_CLOCK_RISING,
RIGHT_DATA_SHIFT} fsm_state = IDLE;
always_ff @(posedge i_clock) begin
case (fsm_state)
IDLE : begin
bit_counter <= \'b0;
shift_register_left <= \'b0;
shift_register_right <= \'b0;
o_codec_dac_data <= \'b0;
if (i_data_valid == 1\'b1) begin
fsm_state <= WAIT_LR_CLOCK_FALLING;
shift_register_left <= i_data_left;
shift_register_right <= i_data_right;
end
end
WAIT_LR_CLOCK_FALLING : begin
if (codec_lr_clock_falling == 1\'b1) begin
fsm_state <= LR_CLOCK_FALLING;
end
end
LR_CLOCK_FALLING : begin
if (codec_bit_clock_rising == 1\'b1) begin
fsm_state <= LEFT_DATA_SHIFT;
end
end
LEFT_DATA_SHIFT : begin
o_codec_dac_data <= shift_register_left[23];
if (codec_bit_clock_rising == 1\'b1) begin
bit_counter <= bit_counter + 1;
if (bit_counter != 0) begin
shift_register_left = {shift_register_left[22:0], 1\'b0};
end;
end
if (bit_counter == 24) begin
bit_counter <= \'b0;
fsm_state <= WAIT_LR_CLOCK_RISING;
end
end
WAIT_LR_CLOCK_RISING : begin
if (codec_lr_clock_rising == 1\'b1) begin
fsm_state <= LR_CLOCK_RISING;
end
end
LR_CLOCK_RISING : begin
if (codec_bit_clock_rising == 1\'b1) begin
fsm_state <= RIGHT_DATA_SHIFT;
end
end
RIGHT_DATA_SHIFT : begin
o_codec_dac_data <= shift_register_right[23];
if (codec_bit_clock_rising == 1\'b1) begin
bit_counter <= bit_counter + 1;
if (bit_counter != 0) begin
shift_register_right = {shift_register_right[22:0], 1\'b0};
end;
end
if (bit_counter == 24) begin
bit_counter <= \'b0;
fsm_state <= IDLE;
end
end
default : begin
fsm_state <= IDLE;
o_codec_dac_data <= 1\'b0;
end
endcase
end
endmoduleInstantiating the Audio (De)Serializer
Now we can replace the audio passthrough from our last post with instances of our Audio (De)Serializer. The Audio Processor core now looks as shown in the figure below.
All our modules for audio processing and analysis will be instantiated between the deserializer and the serializer. The’ll have to comply with the protocol that we have defined:
- Each module shall monitor its ‘i_data_valid‘ input signal. When it is enabled, it indicates that a new sample is available in the ‘i_data_left‘ and ‘i_data_right‘ buses
- Each module shall enable its ‘o_data_valid‘ output signal to indicate that it is done processing the data, and that the processed sample is available in the ‘o_data_left‘ and ‘o_data_right‘ buses.
Rule 2 of our protocol only applies for modules that process data that will be used by downstream logic and will eventually be serialized and sent to the audio codec. Some modules might, say, only run analysis on the audio, and generate non-audio data. This is the case for the LED meter that we will implement in the next post.
Cheers,
Isaac