Audio-XAI

Audio XAI

Python Boilerplate contains all the boilerplate you need to create a Python package.

• GitHub

  PyPI

  Documentation

• Created by Piotr Kitłowski

  GitHub @cncPomper

  PyPI @pkitlo

• MIT License

Features

• TODO

Documentation

Documentation is built with Zensical and deployed to GitHub Pages.

• Live site: https://cncPomper.github.io/Audio-XAI/
• Preview locally: just docs-serve (serves at http://localhost:8000)
• Build: just docs-build

API documentation is auto-generated from docstrings using mkdocstrings.

Docs deploy automatically on push to master via GitHub Actions. To enable this, go to your repo’s Settings > Pages and set the source to GitHub Actions.

Development

To set up for local development:

# Clone your fork
git clone git@github.com:your_username/Audio-XAI.git
cd Audio-XAI

# Install in editable mode with live updates
uv tool install --editable .

This installs the CLI globally but with live updates - any changes you make to the source code are immediately available when you run audio_xai.

Run tests:

uv run pytest

Run quality checks (format, lint, type check, test):

just qa

Author

Audio XAI was created in 2026 by Piotr Kitłowski.

Built with Cookiecutter and the audreyfeldroy/cookiecutter-pypackage project template.

1. General Information and Project Objective

The main objective of the project is to investigate the perceptual fragility of explanations (XAI methods) for deep learning models in the audio domain while keeping predictions unchanged.

2. Planned scope of experiments

• Datasets: Public datasets such as the Speech Commands Dataset (speech) and Sonics (synthetic/real music) will be used. The project will strictly ensure the immutability of the original data.
• Research models: Utilization and adaptation of audio recognition architectures: Audio Spectrogram Transformer, VGGish, Spectra, and ViT.
• XAI methods: Investigation of the vulnerability of gradient-based methods such as Grad-CAM and Integrated Gradients.
• Perceptual constraints: Instead of optimizing attacks against standard metrics, perceptual metrics will be considered (PESQ and STOI for speech, PEAQ for music).
• Computational resources and training: The project will require hardware acceleration (GPUs with a minimum of 16 GB VRAM). The estimated training and fine-tuning time for the base models is approximately 15 hours, while the main process of optimizing perceptual perturbations (XAI attack) for the entire test set is estimated to take an additional 25–30 hours of computation.

3. Planned Program Features

• Classification and Attribution Module: Reading models and generating explanation maps for them.
• Perturbation module: Generating subtle modifications to the audio signal with optimization that preserves high perceptual metrics (PESQ, PEAQ, STOI and ViSQOL).
• Deployment and Automation: Scripted building, testing, and deployment of applications using tools such as just and Python scripts built with typer or argparse.
• Final deliverables: The project will include clear documentation, user instructions, and tests relevant to the project’s scope.

4. Planned Technology Stack

The project will implement a robust base structure, automatically generated by tool cookiecutter.

• Environment management: Use of an isolated virtual environment managed by uv or conda (SLURM-managed cluster will be used).
• Code cleanliness: Enforced PEP8-compliant coding style with an increased line length limit. Syntax checking provided by an autoformatter and a linter using ruff.
• Version control: Rigorous use of a code repository with the conventional commits specification implemented.
• Frameworks and AI: Implementation of learning logic in dedicated frameworks such as PyTorch Lightning in conjunction with Huggingface libraries. Code used for experiments will be continuously exported from Jupyter Lab notebooks into structured library code.
• Experiments and configuration: Tracking progress, metrics, and logs using the Tensorboard platform. The configuration of model parameters and experiments will be completely separated from the execution code.
• Documentation: Use of mkdocs to fast and simple write documentation

5. Project schedule

| Deadline dates in 2026 | Planned scope of work and progress | Status | | :————————: | :— | :—: | | 30.03 - 05.04 | Repository configuration (Cookiecutter, Ruff, Uv). Defining the directory structure and ensuring that audio files remain immutable. | ✔ | | 06.04 - 12.04 | Connecting W&B/TensorBoard. Training base classifiers using the PyTorch Lightning framework. (Estimated resource requirements: 15 hours of GPU computation) | | | 13.04 - 19.04 | Implementation of explanation-generating (XAI) modules in clean code, after first exporting experiments from notebooks. Writing the first tests. | | | 20.04 - 26.04 | Separating configuration from executable code. Preparing baseline attacks on attribution maps using standard distance metrics. | | | 27.04 - 03.05 | Implementation of PESQ/STOI/PEAQ metric approximations directly into the attack optimization loop (generation of perceptual perturbations). | | | 04.05 - 10.05 | Launch of the main research experiments on a dedicated cluster. (Estimated resource requirements: 25–30 hours of GPU computing for iterative processes). | | | 11.05 - 17.05 | Scripting the execution of the entire experiment using the just tool and CLI libraries (e.g., typer). Aggregating tables containing the results. | | | 18.05 - 21.05 | Finalization of the work: creating documentation and clear instructions for using the finished system. Organizing the code in accordance with PEP8. | | | 21.05 - 31.05 | Preparation of the paper(?) | |

This site is open source. Improve this page.