Hi @mravanelli, here's the ltu-as PR as discussed. I am collecting several new datasets and will start a new round of training but this may take time, so meanwhile I start this PR and carry on little by little. @poonehmousavi you are welcome to review the PR as well 😊.

What does this PR do?

1. Add a recipe for training the LTU-AS model (an LLM that jointly understands audio and speech).
2. Slight modifs to the LinearWarmupScheduler class.
3. Adapt the multiwoz llama2 recipe to the latest changes.

To be done

• For now, the model is trained with only half the data than in the paper. Though access to certain datasets seems limited, a new training round needs to be carried out with more datasets being collected.
• Prepare downloadable json files that facilitate the data preparation stage.
• Better to add a tiny validation set for stage 1 and 2.
• An evaluation needs to be implemented at the end of stage 3 and the evaluation data needs to be prepared.
• Upload training logs and prepare a huggingface interface.
• Recipe tests.
• Update results and training details in readme.