speech_dataset_generator_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Speech Dataset Generator - Usage Guide\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pyannote Agreement\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before running the code, ensure that you have agreed to share your contact information to access the pyannote embedding model. A similar agreement may be required for the pyannote speaker diarization model.\n",
    "\n",
    "1. https://huggingface.co/pyannote/embedding\n",
    "\n",
    "2. https://huggingface.co/pyannote/speaker-diarization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Huggingface token"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You need to generate a token at https://huggingface.co/settings/tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Clone the Repository\n",
    "!git clone https://github.com/davidmartinrius/speech-dataset-generator.git\n",
    "%cd speech-dataset-generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Set Up Environment\n",
    "!python3.10 -m venv venv\n",
    "!source venv/bin/activate\n",
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3. HuggingFace Token\n",
    "!echo \"HF_TOKEN=yourtoken\" > .env\n",
    "# Make sure to replace 'yourtoken' with your actual HuggingFace token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3dc9c4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 4. Set up path\n",
    "import os\n",
    "os.environ['PYTHONPATH'] += \":/content/speech-dataset-generator\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from IPython.display import Audio, display\n",
    "\n",
    "def display_the_list_of_files(output_directory):\n",
    "\n",
    "    # List all files in the output directory\n",
    "    file_list = [f for f in os.listdir(output_directory) if f.endswith('.wav')]\n",
    "\n",
    "    # Display the list of files\n",
    "    print(\"List of generated .wav files:\")\n",
    "    for i, file_name in enumerate(file_list[:10]):\n",
    "        print(f\"{file_name}\\n\")\n",
    "        \n",
    "# Function to play audio\n",
    "def play_audio(wavs_directory):\n",
    "\n",
    "    # Let the user choose a file to play\n",
    "    selected_file = input(\"Enter the filename to play (e.g., example_file.wav): \")\n",
    "    file_path = os.path.join(wavs_directory, selected_file)\n",
    "    print(file_path)\n",
    "\n",
    "    # Check if the selected file exists\n",
    "    if os.path.exists(file_path):\n",
    "        print(f\"Playing: {selected_file}\")\n",
    "        display(Audio(filename=file_path))\n",
    "    else:\n",
    "        print(f\"File '{selected_file}' not found in the output directory.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next audio there is:\n",
    "- 2 speakers\n",
    "- 2 genders\n",
    "- Background noise\n",
    "- A length of 2:14 minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Audio(filename=\"./assets/example_audio_1.mp3\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this audio I am going to apply to filters. \n",
    "1. deepfilternet to decrement the noise\n",
    "2. resembleai to enhance the audio quality\n",
    "3. Silence removal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_directory = \"./outputs/output_combining_enhancers\"\n",
    "\n",
    "# No enhancer is used\n",
    "!python speech_dataset_generator/main.py --input_file_path ./assets/example_audio_1.mp3 --output_directory {output_directory} --range_times 3-15 --enhancers deepfilternet resembleai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After processing the audio you got:\n",
    "1- enhanced audios\n",
    "2- Segmented audios in the range you specified. In this case from 5 to 10 seconds for each speaker\n",
    "3- chroma_database, where the speakers are persisted, so you can reuse this database to process other files and the labels of the speakers will be the same\n",
    "4- A metadata.csv + wavs folder, this is the LJSpeech dataset standard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inside enhanced folder you can listen the improved audio without silences: The original was 2:14 minutes. Now it has been reduced to 1:44 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Audio(filename=os.path.join(output_directory, \"enhanced\", \"example_audio_1_enhanced.mp3\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what is inside wavs folder:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wavs_directory = os.path.join(output_directory, \"wavs\")\n",
    "display_the_list_of_files(wavs_directory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Use one of the file names. Example of the output:\n",
    "#List of generated .wav files:\n",
    "#    1709255795_1479612617475313631572.wav\n",
    "\n",
    "#When executing this a prompt will ask for a file name:\n",
    "\n",
    "play_audio(wavs_directory)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced (still in progress)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Example: Input from a File\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Generate with no enhancer. The base audio must be of very good quality, or it will be discarded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_directory = \"./outputs/output_no_enhancer\"\n",
    "\n",
    "# No enhancer is used\n",
    "!python speech_dataset_generator/main.py --input_file_path ./assets/example_audio_1.mp3 --output_directory {output_directory} --range_times 5-10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wavs_directory = os.path.join(output_directory, \"wavs\")\n",
    "display_the_list_of_files(wavs_directory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "play_audio(wavs_directory)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Using deepfilternet enhancer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --input_file_path ./assets/example_audio_1.mp3 --output_directory ./outputs/output_deepfilternet --range_times 4-10 --enhancers deepfilternet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using resembleai enhancer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --input_file_path ./assets/example_audio_1.mp3 --output_directory ./outputs/output_resembleai --range_times 4-10 --enhancers resembleai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Combining enhancers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --input_file_path ./assets/example_audio_1.mp3 --output_directory ./outputs/output_combining_enhancers --range_times 4-10 --enhancers deepfilternet resembleai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Input from a Folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --input_folder ./assets --output_directory ./outputs/output_folder --range_times 4-10 --enhancers deepfilternet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example: Input from YouTube (Single Video or Playlists)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Youtube Single Video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Youtube Single Video\n",
    "!python speech_dataset_generator/main.py --youtube_download https://www.youtube.com/watch?v=ExJZAegsOis --output_directory ./outputs/output_youtube --range_times 5-15 --enhancers deepfilternet resembleai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Combining a YouTube video + Input File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --youtube_download https://www.youtube.com/watch?v=ExJZAegsOis --input_file_path ./assets/example_audio_1.mp3 --output_directory ./outputs/output_youtube_and_file --range_times 5-15 --enhancers deepfilternet resembleai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Combining YouTube video + Input Folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python speech_dataset_generator/main.py --youtube_download https://www.youtube.com/watch?v=ExJZAegsOis --input_folder ./assets --output_directory ./outputs/output_youtube_and_folder --range_times 5-15 --enhancers deepfilternet resembleai"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}