@siteed/expo-audio-studio

Comprehensive audio studio library for React Native and Expo with recording, analysis, visualization, and streaming capabilities across iOS, Android, and web platforms.

iOS Demo

Android Demo

Try AudioPlayground: Complete audio processing app built with this library

Try it in the Playground

Give it a GitHub star 🌟, if you found this repo useful.

Note: This package was formerly known as @siteed/expo-audio-stream. The name has been changed to better reflect the expanded capabilities beyond just audio streaming.

• Real-time audio streaming across iOS, Android, and web.
• Audio input device detection and selection:
  • List and select from available audio input devices
  • View detailed device capabilities (sample rates, channels, bit depths)
  • Support for Bluetooth, USB, and wired devices
  • Automatic device management with fallback options
  • Intelligent detection refresh for Bluetooth devices
• Dual-stream recording capabilities:
  • Simultaneous raw PCM and compressed audio recording
  • Compression formats: OPUS or AAC
  • Configurable bitrate for compressed audio
  • Optimized storage for both high-quality and compressed formats
• Intelligent interruption handling:
  • Automatic pause/resume during phone calls
  • Configurable automatic resumption
  • Detailed interruption event callbacks
• Configurable intervals for audio buffer receipt.
• Automated microphone permissions setup in managed Expo projects.
• Background audio recording on iOS.
• Audio features extraction during recording.
• Consistent WAV PCM recording format across all platforms.
• Keep recording active while app is in background
• Zero-latency recording with preparation API:
  • Pre-initialize audio recording to eliminate startup delay
  • Prepare permissions, audio buffers, and sessions in advance
  • Start recording instantly when needed
• Rich notification system for recording status:
  • Android: Live waveform visualization in notifications
  • Android: Fully customizable notification appearance and actions
  • iOS: Media player integration
• Advanced audio analysis capabilities:
  • Mel spectrogram generation for machine learning and visualization
  • Comprehensive audio feature extraction (MFCC, spectral features, etc.)
  • Lightweight waveform preview generation
• Precision audio manipulation:
  • Advanced audio splitting and trimming API
  • Support for trimming multiple segments in a single operation
  • Ability to keep or remove specific time ranges
• Complete ecosystem:
  • Full-featured AudioPlayground application showcasing advanced API usage
  • Ready-to-use UI components via @siteed/expo-audio-ui package
  • Visualizations, waveforms, and audio controls that can be directly incorporated into your app

Extract powerful audio features for advanced audio processing and visualization:

// Extract audio analysis with specific features enabled
const analysis = await extractAudioAnalysis({
  fileUri: 'path/to/recording.wav',
  features: {
    energy: true,     // Overall energy of the audio
    rms: true,        // Root mean square (amplitude)
    zcr: true,        // Zero-crossing rate
    mfcc: true,       // Mel-frequency cepstral coefficients
    spectralCentroid: true,  // Brightness of sound
    tempo: true,      // Estimated BPM
  }
});

• Basic Analysis: RMS, energy, amplitude range, zero-crossing rate
• Spectral Features: Spectral centroid, flatness, rolloff, bandwidth
• Advanced Analysis:
  • MFCC (Mel-frequency cepstral coefficients)
  • Chromagram (pitch class representation)
  • Mel Spectrogram
  • Harmonics-to-noise ratio
  • Tempo estimation
  • Pitch detection

• Visualize audio waveforms with detailed metrics
• Implement speech recognition preprocessing
• Create music analysis applications
• Build audio fingerprinting systems
• Develop voice activity detection

The library provides several specialized APIs for different audio processing needs:

• useAudioRecorder: Hook for recording audio with configurable quality settings
• AudioRecorderProvider: Context provider for sharing recording state across components
• useSharedAudioRecorder: Hook to access shared recording state from any component

// Start a new recording with configuration
const { startRecording, stopRecording, isRecording, recordingUri } = useAudioRecorder({
  audioQuality: 'high',
  sampleRate: 44100,
  numberOfChannels: 2,
  bitDepth: 16,
  outputFormat: 'wav',
});

// Use the prepare API for zero-latency recording
const { prepareRecording, startRecording, stopRecording } = useSharedAudioRecorder();

// First prepare the recording - this initializes all resources
await prepareRecording({
  sampleRate: 44100,
  channels: 2,
  encoding: 'pcm_16bit'
});

// Later when needed, start instantly with no delay
await startRecording(/* same config as prepare */);

// Share recording state across components
const AudioApp = () => (
  <AudioRecorderProvider>
    <RecordButton />
    <AudioVisualizer />
  </AudioRecorderProvider>
);

• extractAudioAnalysis: Extract comprehensive audio features for detailed analysis
• extractPreview: Generate lightweight waveform data for visualization
• extractAudioData: Extract raw PCM data for custom processing
• extractRawWavAnalysis: Analyze WAV files without decoding, preserving original PCM values

// Extract detailed audio analysis with feature extraction
const analysis = await extractAudioAnalysis({
  fileUri: 'path/to/recording.wav',
  features: { rms: true, zcr: true, mfcc: true }
});

// Generate a lightweight waveform preview
const preview = await extractPreview({
  fileUri: 'path/to/recording.wav',
  pointsPerSecond: 50
});

// Extract raw PCM data for custom processing
const audioData = await extractAudioData({
  fileUri: 'path/to/recording.wav',
  includeWavHeader: true
});

• extractMelSpectrogram: Generate mel spectrogram for audio visualization or ML models
• trimAudio: Trim audio files with precision, supporting multiple segments and formats

// Generate mel spectrogram for audio visualization or ML models
const melSpectrogram = await extractMelSpectrogram({
  fileUri: 'path/to/recording.wav',
  windowSizeMs: 25,
  hopLengthMs: 10,
  nMels: 40
});

// Trim audio files with precision
const trimmedAudio = await trimAudio({
  fileUri: 'path/to/recording.wav',
  startTimeMs: 1000,
  endTimeMs: 5000,
  outputFormat: { format: 'wav' }
});

// Trim multiple segments from an audio file
const compiledAudio = await trimAudio({
  fileUri: 'path/to/recording.wav',
  mode: 'keep',
  ranges: [
    { startTimeMs: 1000, endTimeMs: 5000 },
    { startTimeMs: 10000, endTimeMs: 15000 }
  ]
});

• convertPCMToFloat32: Convert PCM data to Float32Array for processing
• getWavFileInfo: Extract metadata from WAV files
• writeWavHeader: Create WAV headers for raw PCM data

For advanced use cases, the library provides direct access to the native module:

import { ExpoAudioStreamModule } from '@siteed/expo-audio-studio';

// Access platform-specific functionality
const status = await ExpoAudioStreamModule.status();
const permissions = await ExpoAudioStreamModule.getPermissionsAsync();

For detailed documentation, please refer to the Getting Started Guide.

For developers interested in contributing or debugging the library, please see the Contribution Guide.

The repository includes a complete AudioPlayground application that demonstrates advanced usage of the API. This playground serves as both a demonstration and a learning resource:

• Interactive examples of all major API features
• Real-time audio visualization and analysis
• Code samples you can directly reference for your own implementation

Try it online at https://deeeed.github.io/expo-audio-stream/playground or run it locally from the repository.

The @siteed/expo-audio-ui package provides ready-to-use UI components for audio applications:

# Install the UI components package
npm install @siteed/expo-audio-ui

# or with yarn
yarn add @siteed/expo-audio-ui

This package includes:

• Waveform visualizers
• Audio recording controls
• Playback components
• Spectrogram displays
• And more!

All components are built with React Native, Reanimated, and Skia for optimal performance across platforms.

This project is licensed under the MIT License - see the LICENSE file for details.

_{Created by Arthur Breton • See more projects at siteed.net}