Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 31 additions & 11 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,26 @@ Prefer many small, verifiable steps over one large change. Each increment should

If a task feels too large to hold in one session, decompose it further.

### Run Spotless after editing Java

A Spotless formatter is configured for all Java modules. It runs automatically during `install`/`verify`, but a failed format check blocks the build. After editing Java sources run:

```bash
mvn spotless:apply
# or for a single module:
mvn spotless:apply -pl ffsampledsp-java
mvn spotless:apply -pl ffsampledsp-complete
```

---

## Project Overview

FFSampledSP is a Java/JNI library that implements `javax.sound.sampled` service provider interfaces (SPIs) backed by FFmpeg. It decodes audio files/streams to signed linear PCM. Licensed under LGPL 2.1.
FFSampledSP is a Java/JNI library that implements `javax.sound.sampled` service provider interfaces (SPIs) backed by FFmpeg. It decodes audio files/streams to PCM — signed integer (`PCM_SIGNED`) or floating-point (`PCM_FLOAT`). Licensed under LGPL 2.1. Requires Java 8 or later.

## Build Commands

The build requires Maven 3.6+, a JDK, and Doxygen. Native compilation requires platform-specific toolchains. **A platform profile must be activated** — native modules are not built by default.
The build requires Maven 3.6+, a JDK 8+, and Doxygen. Native compilation requires platform-specific toolchains. **A platform profile must be activated** — native modules are not built by default.

### macOS (native for current platform)
```bash
Expand Down Expand Up @@ -75,6 +86,11 @@ mvn --activate-profiles ffsampledsp-aarch64-macos test
mvn --activate-profiles ffsampledsp-aarch64-macos test \
-pl ffsampledsp-complete \
-Dtest=TestFFAudioFileReader

# Run specific test methods
mvn --activate-profiles ffsampledsp-aarch64-macos test \
-pl ffsampledsp-complete \
-Dtest="TestFFCodecInputStream#testConvertWavToFloat32SamplesInRange"
```

### Debug builds
Expand All @@ -85,34 +101,36 @@ Pass `-Dcflags=-DDEBUG` to enable C-level debug output to stdout.
### Module Structure

- **`ffsampledsp-java/`** — Pure Java SPI implementations. The authoritative Java source; compiled standalone but also copied into `ffsampledsp-complete` at build time.
- **`ffsampledsp-{arch}-{host}/`** — Per-platform native modules (`x86_64-macos`, `aarch64-macos`, `x86_64-linux`, `aarch64-linux`, `x86_64-win`, `i386-win`). Each packages a `.dylib`/`.so`/`.dll`. The C sources live only in `ffsampledsp-x86_64-macos/src/main/c/` — all other platform modules symlink or reference the same sources.
- **`ffsampledsp-x86_64-macos/`** — Canonical C source module. All C sources live here; all other platform modules reference the same `src/main/c/` directory via their pom.
- **`ffsampledsp-{arch}-{host}/`** — Per-platform native modules (`aarch64-macos`, `x86_64-linux`, `aarch64-linux`, `x86_64-win`, `i386-win`). Each packages a `.dylib`/`.so`/`.dll` built from the canonical C sources.
- **`ffsampledsp-complete/`** — The distribution artifact. Copies Java sources from `ffsampledsp-java` and embeds the native library from whichever platform profile is active. This is the jar users depend on.

### Java/JNI Layer

The Java classes in `com.tagtraum.ffsampledsp` implement the `javax.sound.sampled.spi` interfaces:

- **`FFAudioFileReader`** — implements `AudioFileReader`. Opens URLs/files/streams via FFmpeg. Caches results (LRU, 20 entries). Has a `getAudioFileFormats()` extension returning multiple formats for multi-stream files (e.g. Stems). Calls into native via two `native` methods: `getAudioFileFormatsFromURL` and `getAudioFileFormatsFromBuffer`.
- **`FFFormatConversionProvider`** — implements `AudioFormatConversionProvider`. Transcodes compressed streams to PCM.
- **`FFAudioFileReader`** — implements `AudioFileReader`. Opens URLs/files/streams via FFmpeg. Caches results (LRU, 20 entries). Has a `getAudioFileFormats()` extension returning multiple formats for multi-stream files (e.g. Stems). The three-argument overloads `getAudioInputStream(file/url, streamIndex, fileBufferSize)` accept an explicit I/O buffer size. Calls into native via two `native` methods: `getAudioFileFormatsFromURL` and `getAudioFileFormatsFromBuffer`.
- **`FFFormatConversionProvider`** — implements `AudioFormatConversionProvider`. Transcodes compressed streams to PCM (`PCM_SIGNED`, `PCM_UNSIGNED`, `PCM_FLOAT`). Standard `AudioFormat.Encoding` constants (e.g. `AudioFormat.Encoding.PCM_FLOAT`) work interchangeably with `FFAudioFormat.FFEncoding` values via string-based resolution. `getAudioInputStream(Encoding, AudioInputStream)` defaults to 32-bit for `PCM_FLOAT` (never inherits a sub-32-bit source depth).
- **`FFNativePeerInputStream`** — abstract base for native-backed `InputStream`s. Holds a `long pointer` to the native C struct and a direct `ByteBuffer` (`nativeBuffer`) that the C side fills.
- **`FFURLInputStream`** — decodes from a URL/file path.
- **`FFURLInputStream`** — decodes from a URL/file path. Configures the FFmpeg I/O buffer size (`AVFormatContext.io_buffer_size`) passed as `fileBufferSize`. Default for `file:` URLs: 1 MB (override with `-Dffsampledsp.fileBufferSize=N`). Default for other URLs: 64 KB (override with `-Dffsampledsp.urlBufferSize=N`).
- **`FFStreamInputStream`** — decodes from a Java `InputStream` (reads into a buffer, probes format, then decodes).
- **`FFCodecInputStream`** — handles format conversion (resampling/channel mapping) using `libswresample`.
- **`FFCodecInputStream`** — handles format conversion (resampling/channel mapping/sample-format) using `libswresample`. Supports `PCM_SIGNED` (8/16/24/32-bit), `PCM_UNSIGNED` (8/16/24/32-bit), and `PCM_FLOAT` (32-bit and 64-bit). Float output is normalized to `[-1, 1]` by libswresample for integer sources; lossy codecs (MP3, AAC) may produce inter-sample peaks slightly outside this range.
- **`FFAudioFormat`** — defines `FFEncoding` (extends `AudioFormat.Encoding`) and the `Codec` enum. All float codec variants (`PCM_F32LE/BE`, `PCM_F64LE/BE`, etc.) use `Encoding.PCM_FLOAT.toString()` as their encoding name — no hardcoded string constant.
- **`FFAudioInputStream`** — wraps an `FFNativePeerInputStream`, implements seeking via `FFGlobalLock`.
- **`FFGlobalLock`** — a single `ReentrantLock` (`LOCK`) used to serialize FFmpeg calls that are not thread-safe (`avcodec_open2`, etc.).
- **`FFNativeLibraryLoader`** — extracts the embedded native library to `java.io.tmpdir` and loads it. Naming convention: `ffsampledsp-{arch}-{host}.{ext}` (e.g. `ffsampledsp-aarch64-macos.dylib`).

All native sources live in one directory — all platforms share them: ffsampledsp-x86_64-macos/src/main/c/
All native sources live in one directory — all platforms share them: `ffsampledsp-x86_64-macos/src/main/c/`

Java language/compiler is specified in the main pom.xml file.
Java language/compiler target is `release=8`, set in the root `pom.xml`.

### C Native Layer (`ffsampledsp-x86_64-macos/src/main/c/`)

- **`FFUtils.c` / `FFUtils.h`** — shared helpers: JNI field/method ID caching, buffer management, FFmpeg context lifecycle, DRM detection (`CODEC_TAG_DRMS`). Minimum probe score of 5 prevents misdetecting files that other `javax.sound.sampled` providers should handle.
- **`FFAudioFileReader.c`** — native implementation of the two `FFAudioFileReader` native methods. Probes format, fills Java `FFAudioFileFormat` / `FFAudioFormat` objects.
- **`FFURLInputStream.c`** — opens an `AVFormatContext` from a URL, decodes packets into the Java `nativeBuffer`.
- **`FFURLInputStream.c`** — opens an `AVFormatContext` from a URL, configures `AVFormatContext.io_buffer_size` from the Java-side `fileBufferSize`, decodes packets into the Java `nativeBuffer`.
- **`FFStreamInputStream.c`** — uses FFmpeg's custom I/O (`AVIOContext` with read callbacks) to pull data from a Java `InputStream`.
- **`FFCodecInputStream.c`** — wraps `libswresample` for PCM conversion.
- **`FFCodecInputStream.c`** — wraps `libswresample` for PCM conversion. Output sample format is selected from the Java-side `AudioFormat`: `AV_SAMPLE_FMT_S16` for 16-bit signed, `AV_SAMPLE_FMT_FLT` for 32-bit float, `AV_SAMPLE_FMT_DBL` for 64-bit float, etc. Uses the modern `av_opt_set_*()` API (not the deprecated `swr_alloc_set_opts()`).

### Key Design Points

Expand All @@ -121,3 +139,5 @@ Java language/compiler is specified in the main pom.xml file.
- Windows URLs require a special format for libav: `file:C:/path/file` (not `file:///C:/path/file`). UNC paths use `file://server/path`. This conversion happens in `FFAudioFileReader.urlToString()`.
- The `fileToURL` method explicitly decodes then re-encodes file URIs to preserve `+` characters in paths (a known edge case).
- JNI headers are auto-generated by `javac -h` during the `compile` phase into `target/native/include/`, then consumed by the platform-specific native module.
- `FFCodecInputStream` uses a `((Buffer) nativeBuffer).limit(0)` cast to work around the covariant `ByteBuffer.limit(int)` return type introduced in Java 9.
- `PCM_FLOAT` support works with the standard `AudioFormat.Encoding.PCM_FLOAT` constant (Java 7+) because all provider methods resolve encodings by calling `FFAudioFormat.FFEncoding.getInstance(encoding.toString())`.
3 changes: 3 additions & 0 deletions NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
- 0.9.55
- Publish to GitHub Pages via GitHub actions.
- Improved `PCM_FLOAT` support: `AudioFormat.Encoding.PCM_FLOAT` now works correctly via
`FFFormatConversionProvider`, defaulting to 32-bit float; 64-bit float decoding is also supported
and tested.


- 0.9.54
Expand Down
61 changes: 54 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,19 @@
# FFSampledSP

*FFSampledSP* is an implementation of the
[javax.sound.sampled](https://docs.oracle.com/javase/10/docs/api/javax/sound/sampled/spi/package-summary.html)
[javax.sound.sampled](https://docs.oracle.com/javase/8/docs/api/javax/sound/sampled/spi/package-summary.html)
service provider interfaces based on [FFmpeg](https://www.ffmpeg.org), a complete, cross-platform solution to record,
convert and stream audio and video.
FFSampledSP is part of the [SampledSP](https://www.tagtraum.com/sampledsp.html) collection of `javax.sound.sampled`
libraries.

Its main purpose is to decode audio files or streams to signed
[linear PCM](https://en.wikipedia.org/wiki/Pulse-code_modulation).
Its main purpose is to decode audio files or streams to
[PCM](https://en.wikipedia.org/wiki/Pulse-code_modulation) — signed integer (`PCM_SIGNED`) or
floating-point (`PCM_FLOAT`).

Supported platforms are currently:

- macOS x64 (>=10.8) and aarch64 (>=11)
- macOS x64 (>=10.10) and aarch64 (>=11)
- Windows i686 and x64
- Linux (Ubuntu 20) x64 and aarch64 (arm64)

Expand All @@ -45,7 +46,7 @@ You can install it via the following dependency:
## Usage

To use the library, simply use
[javax.sound.sampled](https://docs.oracle.com/javase/10/docs/api/javax/sound/sampled/spi/package-summary.html)
[javax.sound.sampled](https://docs.oracle.com/javase/8/docs/api/javax/sound/sampled/spi/package-summary.html)
like you normally would.

Note that opening an `AudioInputStream` of compressed audio (e.g. mp3), does
Expand Down Expand Up @@ -73,17 +74,63 @@ public class DecodeExample {
mp3Format.isBigEndian()
);
// actually decompressed stream (signed PCM)
final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(mp3In, pcmFormat);
final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFormat, mp3In);
// do something with the raw audio stream pcmIn...
}
}
```

To decode to 32-bit float PCM (`PCM_FLOAT`) instead, specify the float encoding and
read the samples via a `FloatBuffer`:

```java
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;

public class FloatDecodeExample {
public static void main(final String[] args) throws Exception {
// compressed stream
final AudioInputStream mp3In = AudioSystem.getAudioInputStream(new File(args[0]));
// AudioFormat describing the compressed stream
final AudioFormat mp3Format = mp3In.getFormat();
// AudioFormat describing 32-bit float PCM output (little-endian, samples in [-1.0, 1.0])
final AudioFormat pcmFloatFormat = new AudioFormat(
AudioFormat.Encoding.PCM_FLOAT,
mp3Format.getSampleRate(),
32,
mp3Format.getChannels(),
32 * mp3Format.getChannels() / 8,
mp3Format.getSampleRate(),
false // little-endian
);
// decoded float PCM stream
final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFloatFormat, mp3In);
// read and process samples as float values
final byte[] buf = new byte[4096];
int justRead;
while ((justRead = pcmIn.read(buf)) != -1) {
final FloatBuffer floats = ByteBuffer.wrap(buf, 0, justRead)
.order(ByteOrder.LITTLE_ENDIAN)
.asFloatBuffer();
while (floats.hasRemaining()) {
final float sample = floats.get(); // value in [-1.0, 1.0]
// process sample...
}
}
}
}
```


## Build

You can build this library locally on macOS, Windows, or Linux (Ubuntu is tested).
When doing so, only the appropriate native libraries are included in the "complete" jar.
When doing so, only the appropriate native libraries are included in the "complete" jar.
The GitHub-based build also adds native libraries for other platforms.

To do so, you also need:
Expand Down
17 changes: 17 additions & 0 deletions ffsampledsp-aarch64-linux/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# ffsampledsp-aarch64-linux

Native library module for Linux aarch64 (arm64). Packages `ffsampledsp-aarch64-linux.so`.

**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there.

## Build

```bash
# Cross-compile from x86_64 Linux (requires aarch64-linux-gnu-gcc):
mvn --activate-profiles ffsampledsp-aarch64-linux install

# Debug build:
mvn --activate-profiles ffsampledsp-aarch64-linux install -Dcflags=-DDEBUG
```

Tests are skipped for this profile (no native runner available in CI).
16 changes: 16 additions & 0 deletions ffsampledsp-aarch64-macos/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# ffsampledsp-aarch64-macos

Native library module for macOS aarch64 (Apple Silicon). Packages `ffsampledsp-aarch64-macos.dylib`.

**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there.

## Build

```bash
mvn --activate-profiles ffsampledsp-aarch64-macos install

# Debug build:
mvn --activate-profiles ffsampledsp-aarch64-macos install -Dcflags=-DDEBUG
```

Requires Apple Command Line Tools or Xcode on Apple Silicon (or cross-compilation from x86_64).
57 changes: 57 additions & 0 deletions ffsampledsp-complete/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# ffsampledsp-complete

Distribution artifact. Bundles:
- Java sources copied from `ffsampledsp-java` (by the Maven resources plugin at build time)
- The native library embedded from whichever platform profile is active

This is the jar users add as a Maven dependency.

## Build

Requires a platform profile to be active (otherwise only Java sources are present, no native lib):

```bash
mvn --activate-profiles ffsampledsp-aarch64-macos install
```

After editing Java sources in `ffsampledsp-java`, fix formatting before the build:

```bash
mvn spotless:apply -pl ffsampledsp-complete
```

## Test Suite

All tests live in `src/test/java/com/tagtraum/ffsampledsp/`. They require the native library to be present (i.e. a platform build must have run first).

| Test class | What it covers |
|---|---|
| `TestAudioSystemIntegration` | End-to-end via `AudioSystem`; includes `testDecodeMp3ToFloatPCM` mirroring the README float sample |
| `TestFFAudioFileReader` | Format detection: encoding, sample rate, channels, frame size, duration, bitrate for all supported formats |
| `TestFFAudioFileFormat` | Supported encoding lookup, codec mapping |
| `TestFFAudioFormat` | `FFEncoding` and `Codec` enum, PCM map, name map |
| `TestFFFormatConversionProvider` | `getTargetEncodings`, `getTargetFormats`, `isConversionSupported`, end-to-end conversions including `PCM_FLOAT` via standard `AudioFormat.Encoding` |
| `TestFFCodecInputStream` | Direct `FFCodecInputStream` conversion tests; PCM_SIGNED, PCM_UNSIGNED, PCM_FLOAT 32-bit and 64-bit with sample-value assertions |
| `TestFFURLInputStream` | URL opening, emoji/special-character paths |
| `TestFFStreamInputStream` | Stream-based decoding, reference sample values |
| `TestFFStreamInputStreamExtended` | Short files, concurrent reads |
| `TestFFNativeLibraryLoader` | Library extraction and loading |

### Test resources

`src/test/resources/com/tagtraum/ffsampledsp/` contains audio files for testing:
- `test.wav` — PCM_SIGNED S16, stereo, 44100 Hz (primary reference file)
- `test24bit.wav` / `test_long24bit.wav` — 24-bit WAV
- `test.mp3` / `test_cbr256.mp3` / `test_vbr130.mp3` — MP3
- `test.flac` / `test24bit.flac` — FLAC (standard and 24-bit)
- `test.m4a` / `test_cbr.m4a` / `test_vbr.m4a` / `test_48k_alac.m4a` — M4A/AAC and ALAC
- `test.w64` — Sony Wave64, PCM_FLOAT 64-bit, stereo 44100 Hz (primary 64-bit float reference)
- `test.ogg` / `test.aiff` / `test.wma` — OGG, AIFF, WMA
- `test_adpcm.wav` — ADPCM WAV
- `test.stem.mp4` — multi-stream Stems file

### PCM_FLOAT test coverage

- **32-bit float**: `testConvertWavToFloat32SamplesInRange`, `testConvertMp3ToFloat32SamplesFiniteAndReasonable`, `testConvertFlac24ToFloat32SamplesInRange` (in `TestFFCodecInputStream`); `testConvertWavToFloat32ViaStandardEncoding` (in `TestFFFormatConversionProvider`)
- **64-bit float**: `testConvertWavToFloat64SamplesInRange`, `testConvertFlac24ToFloat64SamplesInRange` (in `TestFFCodecInputStream`); `testConvertWavToFloat64ViaProvider`, `testReadW64ToFloat64ViaStandardEncoding` (in `TestFFFormatConversionProvider`)
- Float samples from lossless sources (WAV, FLAC) are asserted to be in `[-1.0, 1.0]`; lossy sources (MP3) assert `Float.isFinite` only, as inter-sample peaks slightly above 1.0 are expected
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@

import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import javax.sound.sampled.*;
import org.junit.Test;

Expand Down Expand Up @@ -133,4 +136,44 @@ public void testAudioFileReader3() throws IOException, UnsupportedAudioFileExcep
* targetFormat.getFrameSize());
assertEquals(expectedBytes, bytesRead);
}

@Test
public void testDecodeMp3ToFloatPCM() throws IOException, UnsupportedAudioFileException {
final String filename = "test.mp3";
final File file = File.createTempFile("testDecodeMp3ToFloatPCM", filename);
extractFile(filename, file);
boolean hasNonZero = false;
try (final AudioInputStream mp3In = AudioSystem.getAudioInputStream(file)) {
final AudioFormat mp3Format = mp3In.getFormat();
final AudioFormat pcmFloatFormat =
new AudioFormat(
AudioFormat.Encoding.PCM_FLOAT,
mp3Format.getSampleRate(),
32,
mp3Format.getChannels(),
32 * mp3Format.getChannels() / 8,
mp3Format.getSampleRate(),
false);
try (final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFloatFormat, mp3In)) {
assertEquals(AudioFormat.Encoding.PCM_FLOAT, pcmIn.getFormat().getEncoding());
assertEquals(32, pcmIn.getFormat().getSampleSizeInBits());
final byte[] buf = new byte[4096];
int justRead;
while ((justRead = pcmIn.read(buf)) != -1) {
assertTrue(justRead > 0);
assertEquals("reads must be 4-byte aligned", 0, justRead % 4);
final FloatBuffer floats =
ByteBuffer.wrap(buf, 0, justRead).order(ByteOrder.LITTLE_ENDIAN).asFloatBuffer();
while (floats.hasRemaining()) {
final float sample = floats.get();
assertTrue("Sample must be finite: " + sample, Float.isFinite(sample));
if (sample != 0.0f) hasNonZero = true;
}
}
}
} finally {
file.delete();
}
assertTrue("Expected non-silent audio", hasNonZero);
}
}
Loading
Loading