diff --git a/CLAUDE.md b/CLAUDE.md index ce8ae2f..8033f87 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,15 +27,26 @@ Prefer many small, verifiable steps over one large change. Each increment should If a task feels too large to hold in one session, decompose it further. +### Run Spotless after editing Java + +A Spotless formatter is configured for all Java modules. It runs automatically during `install`/`verify`, but a failed format check blocks the build. After editing Java sources run: + +```bash +mvn spotless:apply +# or for a single module: +mvn spotless:apply -pl ffsampledsp-java +mvn spotless:apply -pl ffsampledsp-complete +``` + --- ## Project Overview -FFSampledSP is a Java/JNI library that implements `javax.sound.sampled` service provider interfaces (SPIs) backed by FFmpeg. It decodes audio files/streams to signed linear PCM. Licensed under LGPL 2.1. +FFSampledSP is a Java/JNI library that implements `javax.sound.sampled` service provider interfaces (SPIs) backed by FFmpeg. It decodes audio files/streams to PCM — signed integer (`PCM_SIGNED`) or floating-point (`PCM_FLOAT`). Licensed under LGPL 2.1. Requires Java 8 or later. ## Build Commands -The build requires Maven 3.6+, a JDK, and Doxygen. Native compilation requires platform-specific toolchains. **A platform profile must be activated** — native modules are not built by default. +The build requires Maven 3.6+, a JDK 8+, and Doxygen. Native compilation requires platform-specific toolchains. **A platform profile must be activated** — native modules are not built by default. ### macOS (native for current platform) ```bash @@ -75,6 +86,11 @@ mvn --activate-profiles ffsampledsp-aarch64-macos test mvn --activate-profiles ffsampledsp-aarch64-macos test \ -pl ffsampledsp-complete \ -Dtest=TestFFAudioFileReader + +# Run specific test methods +mvn --activate-profiles ffsampledsp-aarch64-macos test \ + -pl ffsampledsp-complete \ + -Dtest="TestFFCodecInputStream#testConvertWavToFloat32SamplesInRange" ``` ### Debug builds @@ -85,34 +101,36 @@ Pass `-Dcflags=-DDEBUG` to enable C-level debug output to stdout. ### Module Structure - **`ffsampledsp-java/`** — Pure Java SPI implementations. The authoritative Java source; compiled standalone but also copied into `ffsampledsp-complete` at build time. -- **`ffsampledsp-{arch}-{host}/`** — Per-platform native modules (`x86_64-macos`, `aarch64-macos`, `x86_64-linux`, `aarch64-linux`, `x86_64-win`, `i386-win`). Each packages a `.dylib`/`.so`/`.dll`. The C sources live only in `ffsampledsp-x86_64-macos/src/main/c/` — all other platform modules symlink or reference the same sources. +- **`ffsampledsp-x86_64-macos/`** — Canonical C source module. All C sources live here; all other platform modules reference the same `src/main/c/` directory via their pom. +- **`ffsampledsp-{arch}-{host}/`** — Per-platform native modules (`aarch64-macos`, `x86_64-linux`, `aarch64-linux`, `x86_64-win`, `i386-win`). Each packages a `.dylib`/`.so`/`.dll` built from the canonical C sources. - **`ffsampledsp-complete/`** — The distribution artifact. Copies Java sources from `ffsampledsp-java` and embeds the native library from whichever platform profile is active. This is the jar users depend on. ### Java/JNI Layer The Java classes in `com.tagtraum.ffsampledsp` implement the `javax.sound.sampled.spi` interfaces: -- **`FFAudioFileReader`** — implements `AudioFileReader`. Opens URLs/files/streams via FFmpeg. Caches results (LRU, 20 entries). Has a `getAudioFileFormats()` extension returning multiple formats for multi-stream files (e.g. Stems). Calls into native via two `native` methods: `getAudioFileFormatsFromURL` and `getAudioFileFormatsFromBuffer`. -- **`FFFormatConversionProvider`** — implements `AudioFormatConversionProvider`. Transcodes compressed streams to PCM. +- **`FFAudioFileReader`** — implements `AudioFileReader`. Opens URLs/files/streams via FFmpeg. Caches results (LRU, 20 entries). Has a `getAudioFileFormats()` extension returning multiple formats for multi-stream files (e.g. Stems). The three-argument overloads `getAudioInputStream(file/url, streamIndex, fileBufferSize)` accept an explicit I/O buffer size. Calls into native via two `native` methods: `getAudioFileFormatsFromURL` and `getAudioFileFormatsFromBuffer`. +- **`FFFormatConversionProvider`** — implements `AudioFormatConversionProvider`. Transcodes compressed streams to PCM (`PCM_SIGNED`, `PCM_UNSIGNED`, `PCM_FLOAT`). Standard `AudioFormat.Encoding` constants (e.g. `AudioFormat.Encoding.PCM_FLOAT`) work interchangeably with `FFAudioFormat.FFEncoding` values via string-based resolution. `getAudioInputStream(Encoding, AudioInputStream)` defaults to 32-bit for `PCM_FLOAT` (never inherits a sub-32-bit source depth). - **`FFNativePeerInputStream`** — abstract base for native-backed `InputStream`s. Holds a `long pointer` to the native C struct and a direct `ByteBuffer` (`nativeBuffer`) that the C side fills. - - **`FFURLInputStream`** — decodes from a URL/file path. + - **`FFURLInputStream`** — decodes from a URL/file path. Configures the FFmpeg I/O buffer size (`AVFormatContext.io_buffer_size`) passed as `fileBufferSize`. Default for `file:` URLs: 1 MB (override with `-Dffsampledsp.fileBufferSize=N`). Default for other URLs: 64 KB (override with `-Dffsampledsp.urlBufferSize=N`). - **`FFStreamInputStream`** — decodes from a Java `InputStream` (reads into a buffer, probes format, then decodes). - - **`FFCodecInputStream`** — handles format conversion (resampling/channel mapping) using `libswresample`. + - **`FFCodecInputStream`** — handles format conversion (resampling/channel mapping/sample-format) using `libswresample`. Supports `PCM_SIGNED` (8/16/24/32-bit), `PCM_UNSIGNED` (8/16/24/32-bit), and `PCM_FLOAT` (32-bit and 64-bit). Float output is normalized to `[-1, 1]` by libswresample for integer sources; lossy codecs (MP3, AAC) may produce inter-sample peaks slightly outside this range. +- **`FFAudioFormat`** — defines `FFEncoding` (extends `AudioFormat.Encoding`) and the `Codec` enum. All float codec variants (`PCM_F32LE/BE`, `PCM_F64LE/BE`, etc.) use `Encoding.PCM_FLOAT.toString()` as their encoding name — no hardcoded string constant. - **`FFAudioInputStream`** — wraps an `FFNativePeerInputStream`, implements seeking via `FFGlobalLock`. - **`FFGlobalLock`** — a single `ReentrantLock` (`LOCK`) used to serialize FFmpeg calls that are not thread-safe (`avcodec_open2`, etc.). - **`FFNativeLibraryLoader`** — extracts the embedded native library to `java.io.tmpdir` and loads it. Naming convention: `ffsampledsp-{arch}-{host}.{ext}` (e.g. `ffsampledsp-aarch64-macos.dylib`). -All native sources live in one directory — all platforms share them: ffsampledsp-x86_64-macos/src/main/c/ +All native sources live in one directory — all platforms share them: `ffsampledsp-x86_64-macos/src/main/c/` -Java language/compiler is specified in the main pom.xml file. +Java language/compiler target is `release=8`, set in the root `pom.xml`. ### C Native Layer (`ffsampledsp-x86_64-macos/src/main/c/`) - **`FFUtils.c` / `FFUtils.h`** — shared helpers: JNI field/method ID caching, buffer management, FFmpeg context lifecycle, DRM detection (`CODEC_TAG_DRMS`). Minimum probe score of 5 prevents misdetecting files that other `javax.sound.sampled` providers should handle. - **`FFAudioFileReader.c`** — native implementation of the two `FFAudioFileReader` native methods. Probes format, fills Java `FFAudioFileFormat` / `FFAudioFormat` objects. -- **`FFURLInputStream.c`** — opens an `AVFormatContext` from a URL, decodes packets into the Java `nativeBuffer`. +- **`FFURLInputStream.c`** — opens an `AVFormatContext` from a URL, configures `AVFormatContext.io_buffer_size` from the Java-side `fileBufferSize`, decodes packets into the Java `nativeBuffer`. - **`FFStreamInputStream.c`** — uses FFmpeg's custom I/O (`AVIOContext` with read callbacks) to pull data from a Java `InputStream`. -- **`FFCodecInputStream.c`** — wraps `libswresample` for PCM conversion. +- **`FFCodecInputStream.c`** — wraps `libswresample` for PCM conversion. Output sample format is selected from the Java-side `AudioFormat`: `AV_SAMPLE_FMT_S16` for 16-bit signed, `AV_SAMPLE_FMT_FLT` for 32-bit float, `AV_SAMPLE_FMT_DBL` for 64-bit float, etc. Uses the modern `av_opt_set_*()` API (not the deprecated `swr_alloc_set_opts()`). ### Key Design Points @@ -121,3 +139,5 @@ Java language/compiler is specified in the main pom.xml file. - Windows URLs require a special format for libav: `file:C:/path/file` (not `file:///C:/path/file`). UNC paths use `file://server/path`. This conversion happens in `FFAudioFileReader.urlToString()`. - The `fileToURL` method explicitly decodes then re-encodes file URIs to preserve `+` characters in paths (a known edge case). - JNI headers are auto-generated by `javac -h` during the `compile` phase into `target/native/include/`, then consumed by the platform-specific native module. +- `FFCodecInputStream` uses a `((Buffer) nativeBuffer).limit(0)` cast to work around the covariant `ByteBuffer.limit(int)` return type introduced in Java 9. +- `PCM_FLOAT` support works with the standard `AudioFormat.Encoding.PCM_FLOAT` constant (Java 7+) because all provider methods resolve encodings by calling `FFAudioFormat.FFEncoding.getInstance(encoding.toString())`. diff --git a/NOTES.md b/NOTES.md index cd01aa3..2a82c53 100644 --- a/NOTES.md +++ b/NOTES.md @@ -1,5 +1,8 @@ - 0.9.55 - Publish to GitHub Pages via GitHub actions. + - Improved `PCM_FLOAT` support: `AudioFormat.Encoding.PCM_FLOAT` now works correctly via + `FFFormatConversionProvider`, defaulting to 32-bit float; 64-bit float decoding is also supported + and tested. - 0.9.54 diff --git a/README.md b/README.md index c336c72..d340e70 100644 --- a/README.md +++ b/README.md @@ -7,18 +7,19 @@ # FFSampledSP *FFSampledSP* is an implementation of the -[javax.sound.sampled](https://docs.oracle.com/javase/10/docs/api/javax/sound/sampled/spi/package-summary.html) +[javax.sound.sampled](https://docs.oracle.com/javase/8/docs/api/javax/sound/sampled/spi/package-summary.html) service provider interfaces based on [FFmpeg](https://www.ffmpeg.org), a complete, cross-platform solution to record, convert and stream audio and video. FFSampledSP is part of the [SampledSP](https://www.tagtraum.com/sampledsp.html) collection of `javax.sound.sampled` libraries. -Its main purpose is to decode audio files or streams to signed -[linear PCM](https://en.wikipedia.org/wiki/Pulse-code_modulation). +Its main purpose is to decode audio files or streams to +[PCM](https://en.wikipedia.org/wiki/Pulse-code_modulation) — signed integer (`PCM_SIGNED`) or +floating-point (`PCM_FLOAT`). Supported platforms are currently: -- macOS x64 (>=10.8) and aarch64 (>=11) +- macOS x64 (>=10.10) and aarch64 (>=11) - Windows i686 and x64 - Linux (Ubuntu 20) x64 and aarch64 (arm64) @@ -45,7 +46,7 @@ You can install it via the following dependency: ## Usage To use the library, simply use -[javax.sound.sampled](https://docs.oracle.com/javase/10/docs/api/javax/sound/sampled/spi/package-summary.html) +[javax.sound.sampled](https://docs.oracle.com/javase/8/docs/api/javax/sound/sampled/spi/package-summary.html) like you normally would. Note that opening an `AudioInputStream` of compressed audio (e.g. mp3), does @@ -73,17 +74,63 @@ public class DecodeExample { mp3Format.isBigEndian() ); // actually decompressed stream (signed PCM) - final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(mp3In, pcmFormat); + final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFormat, mp3In); // do something with the raw audio stream pcmIn... } } ``` +To decode to 32-bit float PCM (`PCM_FLOAT`) instead, specify the float encoding and +read the samples via a `FloatBuffer`: + +```java +import java.io.File; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.FloatBuffer; +import javax.sound.sampled.AudioFormat; +import javax.sound.sampled.AudioInputStream; +import javax.sound.sampled.AudioSystem; + +public class FloatDecodeExample { + public static void main(final String[] args) throws Exception { + // compressed stream + final AudioInputStream mp3In = AudioSystem.getAudioInputStream(new File(args[0])); + // AudioFormat describing the compressed stream + final AudioFormat mp3Format = mp3In.getFormat(); + // AudioFormat describing 32-bit float PCM output (little-endian, samples in [-1.0, 1.0]) + final AudioFormat pcmFloatFormat = new AudioFormat( + AudioFormat.Encoding.PCM_FLOAT, + mp3Format.getSampleRate(), + 32, + mp3Format.getChannels(), + 32 * mp3Format.getChannels() / 8, + mp3Format.getSampleRate(), + false // little-endian + ); + // decoded float PCM stream + final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFloatFormat, mp3In); + // read and process samples as float values + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmIn.read(buf)) != -1) { + final FloatBuffer floats = ByteBuffer.wrap(buf, 0, justRead) + .order(ByteOrder.LITTLE_ENDIAN) + .asFloatBuffer(); + while (floats.hasRemaining()) { + final float sample = floats.get(); // value in [-1.0, 1.0] + // process sample... + } + } + } +} +``` + ## Build You can build this library locally on macOS, Windows, or Linux (Ubuntu is tested). -When doing so, only the appropriate native libraries are included in the "complete" jar. +When doing so, only the appropriate native libraries are included in the "complete" jar. The GitHub-based build also adds native libraries for other platforms. To do so, you also need: diff --git a/ffsampledsp-aarch64-linux/CLAUDE.md b/ffsampledsp-aarch64-linux/CLAUDE.md new file mode 100644 index 0000000..3017dc5 --- /dev/null +++ b/ffsampledsp-aarch64-linux/CLAUDE.md @@ -0,0 +1,17 @@ +# ffsampledsp-aarch64-linux + +Native library module for Linux aarch64 (arm64). Packages `ffsampledsp-aarch64-linux.so`. + +**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there. + +## Build + +```bash +# Cross-compile from x86_64 Linux (requires aarch64-linux-gnu-gcc): +mvn --activate-profiles ffsampledsp-aarch64-linux install + +# Debug build: +mvn --activate-profiles ffsampledsp-aarch64-linux install -Dcflags=-DDEBUG +``` + +Tests are skipped for this profile (no native runner available in CI). diff --git a/ffsampledsp-aarch64-macos/CLAUDE.md b/ffsampledsp-aarch64-macos/CLAUDE.md new file mode 100644 index 0000000..8a8ea58 --- /dev/null +++ b/ffsampledsp-aarch64-macos/CLAUDE.md @@ -0,0 +1,16 @@ +# ffsampledsp-aarch64-macos + +Native library module for macOS aarch64 (Apple Silicon). Packages `ffsampledsp-aarch64-macos.dylib`. + +**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there. + +## Build + +```bash +mvn --activate-profiles ffsampledsp-aarch64-macos install + +# Debug build: +mvn --activate-profiles ffsampledsp-aarch64-macos install -Dcflags=-DDEBUG +``` + +Requires Apple Command Line Tools or Xcode on Apple Silicon (or cross-compilation from x86_64). diff --git a/ffsampledsp-complete/CLAUDE.md b/ffsampledsp-complete/CLAUDE.md new file mode 100644 index 0000000..1d3ece4 --- /dev/null +++ b/ffsampledsp-complete/CLAUDE.md @@ -0,0 +1,57 @@ +# ffsampledsp-complete + +Distribution artifact. Bundles: +- Java sources copied from `ffsampledsp-java` (by the Maven resources plugin at build time) +- The native library embedded from whichever platform profile is active + +This is the jar users add as a Maven dependency. + +## Build + +Requires a platform profile to be active (otherwise only Java sources are present, no native lib): + +```bash +mvn --activate-profiles ffsampledsp-aarch64-macos install +``` + +After editing Java sources in `ffsampledsp-java`, fix formatting before the build: + +```bash +mvn spotless:apply -pl ffsampledsp-complete +``` + +## Test Suite + +All tests live in `src/test/java/com/tagtraum/ffsampledsp/`. They require the native library to be present (i.e. a platform build must have run first). + +| Test class | What it covers | +|---|---| +| `TestAudioSystemIntegration` | End-to-end via `AudioSystem`; includes `testDecodeMp3ToFloatPCM` mirroring the README float sample | +| `TestFFAudioFileReader` | Format detection: encoding, sample rate, channels, frame size, duration, bitrate for all supported formats | +| `TestFFAudioFileFormat` | Supported encoding lookup, codec mapping | +| `TestFFAudioFormat` | `FFEncoding` and `Codec` enum, PCM map, name map | +| `TestFFFormatConversionProvider` | `getTargetEncodings`, `getTargetFormats`, `isConversionSupported`, end-to-end conversions including `PCM_FLOAT` via standard `AudioFormat.Encoding` | +| `TestFFCodecInputStream` | Direct `FFCodecInputStream` conversion tests; PCM_SIGNED, PCM_UNSIGNED, PCM_FLOAT 32-bit and 64-bit with sample-value assertions | +| `TestFFURLInputStream` | URL opening, emoji/special-character paths | +| `TestFFStreamInputStream` | Stream-based decoding, reference sample values | +| `TestFFStreamInputStreamExtended` | Short files, concurrent reads | +| `TestFFNativeLibraryLoader` | Library extraction and loading | + +### Test resources + +`src/test/resources/com/tagtraum/ffsampledsp/` contains audio files for testing: +- `test.wav` — PCM_SIGNED S16, stereo, 44100 Hz (primary reference file) +- `test24bit.wav` / `test_long24bit.wav` — 24-bit WAV +- `test.mp3` / `test_cbr256.mp3` / `test_vbr130.mp3` — MP3 +- `test.flac` / `test24bit.flac` — FLAC (standard and 24-bit) +- `test.m4a` / `test_cbr.m4a` / `test_vbr.m4a` / `test_48k_alac.m4a` — M4A/AAC and ALAC +- `test.w64` — Sony Wave64, PCM_FLOAT 64-bit, stereo 44100 Hz (primary 64-bit float reference) +- `test.ogg` / `test.aiff` / `test.wma` — OGG, AIFF, WMA +- `test_adpcm.wav` — ADPCM WAV +- `test.stem.mp4` — multi-stream Stems file + +### PCM_FLOAT test coverage + +- **32-bit float**: `testConvertWavToFloat32SamplesInRange`, `testConvertMp3ToFloat32SamplesFiniteAndReasonable`, `testConvertFlac24ToFloat32SamplesInRange` (in `TestFFCodecInputStream`); `testConvertWavToFloat32ViaStandardEncoding` (in `TestFFFormatConversionProvider`) +- **64-bit float**: `testConvertWavToFloat64SamplesInRange`, `testConvertFlac24ToFloat64SamplesInRange` (in `TestFFCodecInputStream`); `testConvertWavToFloat64ViaProvider`, `testReadW64ToFloat64ViaStandardEncoding` (in `TestFFFormatConversionProvider`) +- Float samples from lossless sources (WAV, FLAC) are asserted to be in `[-1.0, 1.0]`; lossy sources (MP3) assert `Float.isFinite` only, as inter-sample peaks slightly above 1.0 are expected diff --git a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestAudioSystemIntegration.java b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestAudioSystemIntegration.java index fddc3f9..8fc5d52 100644 --- a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestAudioSystemIntegration.java +++ b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestAudioSystemIntegration.java @@ -27,6 +27,9 @@ import java.io.File; import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.FloatBuffer; import javax.sound.sampled.*; import org.junit.Test; @@ -133,4 +136,44 @@ public void testAudioFileReader3() throws IOException, UnsupportedAudioFileExcep * targetFormat.getFrameSize()); assertEquals(expectedBytes, bytesRead); } + + @Test + public void testDecodeMp3ToFloatPCM() throws IOException, UnsupportedAudioFileException { + final String filename = "test.mp3"; + final File file = File.createTempFile("testDecodeMp3ToFloatPCM", filename); + extractFile(filename, file); + boolean hasNonZero = false; + try (final AudioInputStream mp3In = AudioSystem.getAudioInputStream(file)) { + final AudioFormat mp3Format = mp3In.getFormat(); + final AudioFormat pcmFloatFormat = + new AudioFormat( + AudioFormat.Encoding.PCM_FLOAT, + mp3Format.getSampleRate(), + 32, + mp3Format.getChannels(), + 32 * mp3Format.getChannels() / 8, + mp3Format.getSampleRate(), + false); + try (final AudioInputStream pcmIn = AudioSystem.getAudioInputStream(pcmFloatFormat, mp3In)) { + assertEquals(AudioFormat.Encoding.PCM_FLOAT, pcmIn.getFormat().getEncoding()); + assertEquals(32, pcmIn.getFormat().getSampleSizeInBits()); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmIn.read(buf)) != -1) { + assertTrue(justRead > 0); + assertEquals("reads must be 4-byte aligned", 0, justRead % 4); + final FloatBuffer floats = + ByteBuffer.wrap(buf, 0, justRead).order(ByteOrder.LITTLE_ENDIAN).asFloatBuffer(); + while (floats.hasRemaining()) { + final float sample = floats.get(); + assertTrue("Sample must be finite: " + sample, Float.isFinite(sample)); + if (sample != 0.0f) hasNonZero = true; + } + } + } + } finally { + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } } diff --git a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFCodecInputStream.java b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFCodecInputStream.java index 758a48b..1cb0c46 100644 --- a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFCodecInputStream.java +++ b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFCodecInputStream.java @@ -23,6 +23,10 @@ import static org.junit.Assert.*; import java.io.*; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.DoubleBuffer; +import java.nio.FloatBuffer; import java.util.concurrent.TimeUnit; import javax.sound.sampled.AudioFormat; import javax.sound.sampled.AudioInputStream; @@ -886,6 +890,176 @@ public void testReadConvertWaveStreamToFloatPCM() assertEquals(1069056, bytesRead); } + @Test + public void testConvertWavToFloat32SamplesInRange() + throws IOException, UnsupportedAudioFileException { + final String filename = "test.wav"; + final File file = File.createTempFile("testConvertWavToFloat32SamplesInRange", filename); + extractFile(filename, file); + boolean hasNonZero = false; + FFCodecInputStream pcmStream = null; + try (final AudioInputStream sourceStream = new FFAudioFileReader().getAudioInputStream(file)) { + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 32, 2, 8, 44100, false); + final ByteOrder byteOrder = + targetFormat.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + pcmStream = new FFCodecInputStream(targetFormat, (FFAudioInputStream) sourceStream); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmStream.read(buf)) != -1) { + assertEquals("reads must be 4-byte aligned", 0, justRead % 4); + final FloatBuffer floats = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asFloatBuffer(); + while (floats.hasRemaining()) { + final float s = floats.get(); + assertTrue("Sample out of range [-1,1]: " + s, s >= -1.0f && s <= 1.0f); + if (s != 0.0f) hasNonZero = true; + } + } + } finally { + if (pcmStream != null) pcmStream.close(); + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } + + @Test + public void testConvertMp3ToFloat32SamplesFiniteAndReasonable() + throws IOException, UnsupportedAudioFileException { + // MP3 reconstruction can produce inter-sample peaks slightly outside [-1, 1] — this is + // expected behaviour (same as libsndfile, dr_mp3, etc.), not a bug. We verify samples are + // finite and within a small headroom rather than asserting strict [-1, 1]. + final String filename = "test.mp3"; + final File file = + File.createTempFile("testConvertMp3ToFloat32SamplesFiniteAndReasonable", filename); + extractFile(filename, file); + boolean hasNonZero = false; + FFCodecInputStream pcmStream = null; + try (final AudioInputStream sourceStream = new FFAudioFileReader().getAudioInputStream(file)) { + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 32, 2, 8, 44100, false); + final ByteOrder byteOrder = + targetFormat.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + pcmStream = new FFCodecInputStream(targetFormat, (FFAudioInputStream) sourceStream); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmStream.read(buf)) != -1) { + assertEquals("reads must be 4-byte aligned", 0, justRead % 4); + final FloatBuffer floats = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asFloatBuffer(); + while (floats.hasRemaining()) { + final float s = floats.get(); + assertTrue("Sample must be finite: " + s, Float.isFinite(s)); + assertTrue("Sample implausibly large: " + s, s >= -2.0f && s <= 2.0f); + if (s != 0.0f) hasNonZero = true; + } + } + } finally { + if (pcmStream != null) pcmStream.close(); + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } + + @Test + public void testConvertFlac24ToFloat32SamplesInRange() + throws IOException, UnsupportedAudioFileException { + final String filename = "test24bit.flac"; + final File file = File.createTempFile("testConvertFlac24ToFloat32SamplesInRange", filename); + extractFile(filename, file); + boolean hasNonZero = false; + FFCodecInputStream pcmStream = null; + try (final AudioInputStream sourceStream = new FFAudioFileReader().getAudioInputStream(file)) { + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 32, 2, 8, 44100, false); + final ByteOrder byteOrder = + targetFormat.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + pcmStream = new FFCodecInputStream(targetFormat, (FFAudioInputStream) sourceStream); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmStream.read(buf)) != -1) { + assertEquals("reads must be 4-byte aligned", 0, justRead % 4); + final FloatBuffer floats = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asFloatBuffer(); + while (floats.hasRemaining()) { + final float s = floats.get(); + assertTrue("Sample out of range [-1,1]: " + s, s >= -1.0f && s <= 1.0f); + if (s != 0.0f) hasNonZero = true; + } + } + } finally { + if (pcmStream != null) pcmStream.close(); + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } + + @Test + public void testConvertWavToFloat64SamplesInRange() + throws IOException, UnsupportedAudioFileException { + final String filename = "test.wav"; + final File file = File.createTempFile("testConvertWavToFloat64SamplesInRange", filename); + extractFile(filename, file); + boolean hasNonZero = false; + FFCodecInputStream pcmStream = null; + try (final AudioInputStream sourceStream = new FFAudioFileReader().getAudioInputStream(file)) { + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 64, 2, 16, 44100, false); + final ByteOrder byteOrder = + targetFormat.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + pcmStream = new FFCodecInputStream(targetFormat, (FFAudioInputStream) sourceStream); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmStream.read(buf)) != -1) { + assertEquals("reads must be 8-byte aligned", 0, justRead % 8); + final DoubleBuffer doubles = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asDoubleBuffer(); + while (doubles.hasRemaining()) { + final double d = doubles.get(); + assertTrue("Sample out of range [-1,1]: " + d, d >= -1.0 && d <= 1.0); + if (d != 0.0) hasNonZero = true; + } + } + } finally { + if (pcmStream != null) pcmStream.close(); + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } + + @Test + public void testConvertFlac24ToFloat64SamplesInRange() + throws IOException, UnsupportedAudioFileException { + final String filename = "test24bit.flac"; + final File file = File.createTempFile("testConvertFlac24ToFloat64SamplesInRange", filename); + extractFile(filename, file); + boolean hasNonZero = false; + FFCodecInputStream pcmStream = null; + try (final AudioInputStream sourceStream = new FFAudioFileReader().getAudioInputStream(file)) { + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 64, 2, 16, 44100, false); + final ByteOrder byteOrder = + targetFormat.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + pcmStream = new FFCodecInputStream(targetFormat, (FFAudioInputStream) sourceStream); + final byte[] buf = new byte[4096]; + int justRead; + while ((justRead = pcmStream.read(buf)) != -1) { + assertEquals("reads must be 8-byte aligned", 0, justRead % 8); + final DoubleBuffer doubles = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asDoubleBuffer(); + while (doubles.hasRemaining()) { + final double d = doubles.get(); + assertTrue("Sample out of range [-1,1]: " + d, d >= -1.0 && d <= 1.0); + if (d != 0.0) hasNonZero = true; + } + } + } finally { + if (pcmStream != null) pcmStream.close(); + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + } + @Test public void testConvertWaveFileTo24bit() throws IOException, UnsupportedAudioFileException { final String filename = "test.wav"; diff --git a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFFormatConversionProvider.java b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFFormatConversionProvider.java index af489cc..7169b9e 100644 --- a/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFFormatConversionProvider.java +++ b/ffsampledsp-complete/src/test/java/com/tagtraum/ffsampledsp/TestFFFormatConversionProvider.java @@ -26,6 +26,10 @@ import java.io.File; import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.DoubleBuffer; +import java.nio.FloatBuffer; import java.util.Arrays; import java.util.HashSet; import java.util.Set; @@ -365,6 +369,229 @@ public void testGetUnsupportedTargetFormats() { assertEquals(0, targetFormatsEncodingAAC.length); } + @Test + public void testIsConversionSupportedStandardPCMFloat() + throws IOException, UnsupportedAudioFileException { + final String filename = "test.wav"; + final File file = File.createTempFile("testIsConversionSupportedStandardPCMFloat", filename); + extractFile(filename, file); + try { + final AudioFormat sourceFormat = + new FFAudioFileReader().getAudioInputStream(file).getFormat(); + assertTrue( + new FFFormatConversionProvider() + .isConversionSupported(AudioFormat.Encoding.PCM_FLOAT, sourceFormat)); + } finally { + file.delete(); + } + } + + @Test + public void testGetTargetFormatsStandardPCMFloat() { + final AudioFormat[] targetFormats = + new FFFormatConversionProvider() + .getTargetFormats( + AudioFormat.Encoding.PCM_FLOAT, + new AudioFormat( + FFAudioFormat.FFEncoding.Codec.MP3.getEncoding(), + 22050f, + 16, + STEREO, + AudioSystem.NOT_SPECIFIED, + AudioSystem.NOT_SPECIFIED, + true)); + assertEquals(4, targetFormats.length); + final Set audioFormats = new HashSet<>(Arrays.asList(targetFormats)); + assertTrue( + containsWithMatches( + audioFormats, + new AudioFormat( + AudioFormat.Encoding.PCM_FLOAT, + AudioSystem.NOT_SPECIFIED, + 32, + MONO, + 4, + AudioSystem.NOT_SPECIFIED, + NATIVE_ORDER))); + assertTrue( + containsWithMatches( + audioFormats, + new AudioFormat( + AudioFormat.Encoding.PCM_FLOAT, + AudioSystem.NOT_SPECIFIED, + 32, + STEREO, + 8, + AudioSystem.NOT_SPECIFIED, + NATIVE_ORDER))); + } + + @Test + public void testConvertWavToFloat32ViaStandardEncoding() + throws IOException, UnsupportedAudioFileException { + final String filename = "test.wav"; + final File file = File.createTempFile("testConvertWavToFloat32ViaStandardEncoding", filename); + extractFile(filename, file); + int bytesRead = 0; + boolean hasNonZero = false; + AudioInputStream in = null; + AudioInputStream convertedIn = null; + try { + in = new FFAudioFileReader().getAudioInputStream(file); + convertedIn = + new FFFormatConversionProvider().getAudioInputStream(AudioFormat.Encoding.PCM_FLOAT, in); + final AudioFormat fmt = convertedIn.getFormat(); + assertEquals(AudioFormat.Encoding.PCM_FLOAT, fmt.getEncoding()); + assertEquals(32, fmt.getSampleSizeInBits()); + final ByteOrder byteOrder = + fmt.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + int justRead; + final byte[] buf = new byte[4096]; + while ((justRead = convertedIn.read(buf)) != -1) { + assertTrue(justRead > 0); + bytesRead += justRead; + assertEquals("reads must be 4-byte aligned", 0, justRead % 4); + final FloatBuffer floats = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asFloatBuffer(); + while (floats.hasRemaining()) { + final float s = floats.get(); + assertTrue("Sample out of range [-1,1]: " + s, s >= -1.0f && s <= 1.0f); + if (s != 0.0f) hasNonZero = true; + } + } + } finally { + if (convertedIn != null) { + try { + convertedIn.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + if (in != null) { + try { + in.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + assertEquals(1069056, bytesRead); + } + + @Test + public void testConvertWavToFloat64ViaProvider() + throws IOException, UnsupportedAudioFileException { + final String filename = "test.wav"; + final File file = File.createTempFile("testConvertWavToFloat64ViaProvider", filename); + extractFile(filename, file); + int bytesRead = 0; + boolean hasNonZero = false; + AudioInputStream in = null; + AudioInputStream convertedIn = null; + try { + in = new FFAudioFileReader().getAudioInputStream(file); + final AudioFormat targetFormat = + new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 44100, 64, 2, 16, 44100, false); + convertedIn = new FFFormatConversionProvider().getAudioInputStream(targetFormat, in); + final AudioFormat fmt = convertedIn.getFormat(); + assertEquals(AudioFormat.Encoding.PCM_FLOAT, fmt.getEncoding()); + assertEquals(64, fmt.getSampleSizeInBits()); + final ByteOrder byteOrder = + fmt.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + int justRead; + final byte[] buf = new byte[4096]; + while ((justRead = convertedIn.read(buf)) != -1) { + assertTrue(justRead > 0); + bytesRead += justRead; + assertEquals("reads must be 8-byte aligned", 0, justRead % 8); + final DoubleBuffer doubles = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asDoubleBuffer(); + while (doubles.hasRemaining()) { + final double d = doubles.get(); + assertTrue("Sample out of range [-1,1]: " + d, d >= -1.0 && d <= 1.0); + if (d != 0.0) hasNonZero = true; + } + } + } finally { + if (convertedIn != null) { + try { + convertedIn.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + if (in != null) { + try { + in.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + assertEquals(2138112, bytesRead); + } + + @Test + public void testReadW64ToFloat64ViaStandardEncoding() + throws IOException, UnsupportedAudioFileException { + // test.w64 is natively PCM_FLOAT 64-bit; the encoding-only path should inherit the 64-bit + // depth and produce a 64-bit output stream. The file's float samples are not normalized to + // [-1,1] (valid per the IEEE float PCM convention), so we assert finiteness rather than range. + final String filename = "test.w64"; + final File file = File.createTempFile("testReadW64ToFloat64ViaStandardEncoding", filename); + extractFile(filename, file); + int bytesRead = 0; + boolean hasNonZero = false; + AudioInputStream in = null; + AudioInputStream convertedIn = null; + try { + in = new FFAudioFileReader().getAudioInputStream(file); + convertedIn = + new FFFormatConversionProvider().getAudioInputStream(AudioFormat.Encoding.PCM_FLOAT, in); + final AudioFormat fmt = convertedIn.getFormat(); + assertEquals(AudioFormat.Encoding.PCM_FLOAT, fmt.getEncoding()); + assertEquals(64, fmt.getSampleSizeInBits()); + final ByteOrder byteOrder = + fmt.isBigEndian() ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN; + int justRead; + final byte[] buf = new byte[4096]; + while ((justRead = convertedIn.read(buf)) != -1) { + assertTrue(justRead > 0); + bytesRead += justRead; + assertEquals("reads must be 8-byte aligned", 0, justRead % 8); + final DoubleBuffer doubles = + ByteBuffer.wrap(buf, 0, justRead).order(byteOrder).asDoubleBuffer(); + while (doubles.hasRemaining()) { + final double d = doubles.get(); + assertTrue("Sample must be finite: " + d, Double.isFinite(d)); + if (d != 0.0) hasNonZero = true; + } + } + } finally { + if (convertedIn != null) { + try { + convertedIn.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + if (in != null) { + try { + in.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + file.delete(); + } + assertTrue("Expected non-silent audio", hasNonZero); + assertEquals(2138112, bytesRead); + } + /** * AudioFormat uses {@link AudioFormat#matches(javax.sound.sampled.AudioFormat)} instead of {@link * Object#equals(Object)} for comparison - therefore we need a special contains() diff --git a/ffsampledsp-i386-win/CLAUDE.md b/ffsampledsp-i386-win/CLAUDE.md new file mode 100644 index 0000000..12c6f55 --- /dev/null +++ b/ffsampledsp-i386-win/CLAUDE.md @@ -0,0 +1,14 @@ +# ffsampledsp-i386-win + +Native library module for Windows i386 (32-bit). Packages `ffsampledsp-i386-win.dll`. + +**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there. + +## Build + +```bash +# Requires MSYS2 with MinGW 32-bit GCC toolchain: +mvn --activate-profiles ffsampledsp-i386-win install +``` + +Tests are skipped for this profile. diff --git a/ffsampledsp-java/CLAUDE.md b/ffsampledsp-java/CLAUDE.md new file mode 100644 index 0000000..e72c6ae --- /dev/null +++ b/ffsampledsp-java/CLAUDE.md @@ -0,0 +1,39 @@ +# ffsampledsp-java + +Authoritative Java SPI implementation module. Contains all `com.tagtraum.ffsampledsp` Java sources. Compiled standalone; sources are also **copied** into `ffsampledsp-complete` at build time by the Maven resources plugin. + +## Build + +```bash +# Java-only (no native): +mvn install -pl ffsampledsp-java + +# After editing Java: fix formatting before the build checks it: +mvn spotless:apply -pl ffsampledsp-java +``` + +Java compiler target is `8`, set in the root `pom.xml`. + +## Key Classes + +| Class | Role | +|---|---| +| `FFAudioFileReader` | `AudioFileReader` SPI. Opens URLs/files/streams; LRU cache (20 entries). `getAudioInputStream(…, fileBufferSize)` overloads accept an explicit FFmpeg I/O buffer size. | +| `FFFormatConversionProvider` | `FormatConversionProvider` SPI. Converts to `PCM_SIGNED`, `PCM_UNSIGNED`, or `PCM_FLOAT`. Standard `AudioFormat.Encoding` constants work via string-based resolution. `getAudioInputStream(Encoding, stream)` defaults to 32-bit for `PCM_FLOAT`. | +| `FFURLInputStream` | Native-backed stream from URL/file. Passes `fileBufferSize` to `AVFormatContext.io_buffer_size`. Default: 1 MB for `file:` URLs (`-Dffsampledsp.fileBufferSize=N`), 64 KB for others (`-Dffsampledsp.urlBufferSize=N`). | +| `FFStreamInputStream` | Native-backed stream from a Java `InputStream`. | +| `FFCodecInputStream` | libswresample-backed format converter. Supports `PCM_SIGNED`/`PCM_UNSIGNED` (8–32-bit) and `PCM_FLOAT` (32-bit and 64-bit). | +| `FFAudioFormat` | Defines `FFEncoding` and the `Codec` enum mapping FFmpeg codec IDs to Java encodings. All float variants use `Encoding.PCM_FLOAT.toString()` — no hardcoded `"PCM_FLOAT"` string. | +| `FFAudioInputStream` | Wraps `FFNativePeerInputStream`; implements seeking. | +| `FFGlobalLock` | Single `ReentrantLock` serialising non-thread-safe FFmpeg API calls. | +| `FFNativeLibraryLoader` | Extracts embedded `.dylib`/`.so`/`.dll` to `java.io.tmpdir` and loads it. | + +## PCM_FLOAT Notes + +- `AudioFormat.Encoding.PCM_FLOAT` (standard Java 7+) and `FFAudioFormat.FFEncoding.PCM_FLOAT` are interchangeable — all provider methods resolve via `getInstance(encoding.toString())`. +- `getAudioInputStream(Encoding, AudioInputStream)` in `FFFormatConversionProvider` enforces a minimum of 32 bits for `PCM_FLOAT` targets (a 16-bit source must not bleed through). +- Integer-source float output is normalised to `[-1, 1]` by libswresample. Lossy codecs (MP3, AAC) may produce inter-sample peaks slightly outside this range — this is expected behaviour, not a bug. + +## JNI Interface + +Native methods are declared in `FFAudioFileReader` (`getAudioFileFormatsFromURL`, `getAudioFileFormatsFromBuffer`) and `FFNativePeerInputStream` subclasses. JNI headers are generated by `javac -h` into `target/native/include/` during the `compile` phase and consumed by the platform native module. diff --git a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFileReader.java b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFileReader.java index ecbcf63..c89cd52 100644 --- a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFileReader.java +++ b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFileReader.java @@ -29,6 +29,7 @@ import java.nio.ByteBuffer; import java.nio.channels.Channels; import java.nio.channels.ReadableByteChannel; +import java.nio.charset.StandardCharsets; import java.util.Collections; import java.util.LinkedHashMap; import java.util.Map; @@ -398,7 +399,8 @@ private AudioFileFormat[] lockedGetAudioFileFormatsFromURL(final String url) throws IOException, UnsupportedAudioFileException { LOCK.lock(); try { - final AudioFileFormat[] audioFileFormat = getAudioFileFormatsFromURL(url); + final AudioFileFormat[] audioFileFormat = + getAudioFileFormatsFromURL(url, url.getBytes(StandardCharsets.UTF_8)); checkPlausibility(audioFileFormat); return audioFileFormat; } finally { @@ -434,8 +436,8 @@ private AudioFileFormat[] lockedGetAudioFileFormatFromBuffer(final ByteBuffer by * @return {@link AudioFileFormat}s * @throws IOException if an IO error occurs */ - private native AudioFileFormat[] getAudioFileFormatsFromURL(final String url) - throws IOException, UnsupportedAudioFileException; + private native AudioFileFormat[] getAudioFileFormatsFromURL( + final String url, final byte[] urlBytes) throws IOException, UnsupportedAudioFileException; /** * Determine {@link AudioFileFormat} from a file containing just the first kbs from a stream. diff --git a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFormat.java b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFormat.java index 92a7d8a..5edc928 100644 --- a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFormat.java +++ b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFAudioFormat.java @@ -330,13 +330,6 @@ public static class FFEncoding extends Encoding { private static final int AV_CODEC_ID_G728 = 0x1506b; private static final int AV_CODEC_ID_AHX = 0x1506c; - /** - * Float PCM - named just like the PCM_FLOAT in {@link - * javax.sound.sampled.AudioFormat.Encoding}. in Java 7 (not used for compatibility with Java - * <7). - */ - private static final String PCM_FLOAT_STRING = "PCM_FLOAT"; - /** Codecs supported by libavcodec. */ public enum Codec { /** MPEG-1 Layer 1 audio. */ @@ -371,7 +364,7 @@ public enum Codec { /** Generic unsigned PCM (format determined by sample size and endianness). */ PCM_UNSIGNED(Encoding.PCM_UNSIGNED.toString(), -1, true), /** Generic floating-point PCM (format determined by sample size and endianness). */ - PCM_FLOAT(PCM_FLOAT_STRING, -1, true), + PCM_FLOAT(Encoding.PCM_FLOAT.toString(), -1, true), /** Signed 8-bit PCM. */ PCM_S8(Encoding.PCM_SIGNED.toString(), AV_CODEC_ID_PCM_S8, true), @@ -408,17 +401,17 @@ public enum Codec { PCM_U32LE(Encoding.PCM_UNSIGNED.toString(), AV_CODEC_ID_PCM_U32LE, true), /** 16-bit floating-point little-endian PCM. */ - PCM_F16LE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F16LE, true), + PCM_F16LE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F16LE, true), /** 24-bit floating-point little-endian PCM. */ - PCM_F24LE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F24LE, true), + PCM_F24LE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F24LE, true), /** 32-bit floating-point big-endian PCM. */ - PCM_F32BE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F32BE, true), + PCM_F32BE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F32BE, true), /** 32-bit floating-point little-endian PCM. */ - PCM_F32LE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F32LE, true), + PCM_F32LE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F32LE, true), /** 64-bit floating-point big-endian PCM. */ - PCM_F64BE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F64BE, true), + PCM_F64BE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F64BE, true), /** 64-bit floating-point little-endian PCM. */ - PCM_F64LE(PCM_FLOAT_STRING, AV_CODEC_ID_PCM_F64LE, true), + PCM_F64LE(Encoding.PCM_FLOAT.toString(), AV_CODEC_ID_PCM_F64LE, true), /** Signed 8-bit planar PCM. */ PCM_S8_PLANAR("PCM S8 Planar", AV_CODEC_ID_PCM_S8_PLANAR, true), diff --git a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFFormatConversionProvider.java b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFFormatConversionProvider.java index f2e1def..4c5c1e4 100644 --- a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFFormatConversionProvider.java +++ b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFFormatConversionProvider.java @@ -252,13 +252,18 @@ public AudioInputStream getAudioInputStream( public AudioInputStream getAudioInputStream( final AudioFormat.Encoding targetEncoding, final AudioInputStream sourceStream) { final AudioFormat sourceFormat = sourceStream.getFormat(); - // we assume some defaults... - final int sampleSizeInBits = - sourceFormat.getSampleSizeInBits() > 0 ? sourceFormat.getSampleSizeInBits() : 16; - final int frameSize = - sourceFormat.getFrameSize() > 0 - ? sourceFormat.getFrameSize() - : sampleSizeInBits * sourceFormat.getChannels() / 8; + final FFAudioFormat.FFEncoding ffEncoding = + FFAudioFormat.FFEncoding.getInstance(targetEncoding.toString()); + // PCM_FLOAT requires at least 32 bits; a source like S16 must not be inherited as-is + final int sampleSizeInBits; + if (PCM_FLOAT.getEncoding().equals(ffEncoding)) { + final int srcBits = sourceFormat.getSampleSizeInBits(); + sampleSizeInBits = srcBits >= 32 ? srcBits : 32; + } else { + sampleSizeInBits = + sourceFormat.getSampleSizeInBits() > 0 ? sourceFormat.getSampleSizeInBits() : 16; + } + final int frameSize = sampleSizeInBits * sourceFormat.getChannels() / 8; final AudioFormat targetFormat = new AudioFormat( targetEncoding, diff --git a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFURLInputStream.java b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFURLInputStream.java index 2703603..d8f5ece 100644 --- a/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFURLInputStream.java +++ b/ffsampledsp-java/src/main/java/com/tagtraum/ffsampledsp/FFURLInputStream.java @@ -25,6 +25,7 @@ import java.io.IOException; import java.net.URL; import java.nio.Buffer; +import java.nio.charset.StandardCharsets; import java.util.concurrent.TimeUnit; import javax.sound.sampled.UnsupportedAudioFileException; @@ -251,7 +252,7 @@ private long lockedOpen(final String url, final int streamIndex, final int fileB throws IOException, UnsupportedAudioFileException { LOCK.lock(); try { - return open(url, streamIndex, fileBufferSize); + return open(url, url.getBytes(StandardCharsets.UTF_8), streamIndex, fileBufferSize); } finally { LOCK.unlock(); } @@ -263,7 +264,8 @@ private long lockedOpen(final String url, final int streamIndex, final int fileB private native void fillNativeBuffer(final long pointer) throws IOException; - private native long open(final String url, final int streamIndex, final int fileBufferSize) + private native long open( + final String url, final byte[] urlBytes, final int streamIndex, final int fileBufferSize) throws IOException, UnsupportedAudioFileException; protected native void close(final long pointer) throws IOException; diff --git a/ffsampledsp-x86_64-linux/CLAUDE.md b/ffsampledsp-x86_64-linux/CLAUDE.md new file mode 100644 index 0000000..48abf57 --- /dev/null +++ b/ffsampledsp-x86_64-linux/CLAUDE.md @@ -0,0 +1,16 @@ +# ffsampledsp-x86_64-linux + +Native library module for Linux x86_64. Packages `ffsampledsp-x86_64-linux.so`. + +**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there. + +## Build + +```bash +mvn --activate-profiles ffsampledsp-x86_64-linux install + +# Debug build: +mvn --activate-profiles ffsampledsp-x86_64-linux install -Dcflags=-DDEBUG +``` + +Requires GCC. Tested on Ubuntu 20+. diff --git a/ffsampledsp-x86_64-macos/CLAUDE.md b/ffsampledsp-x86_64-macos/CLAUDE.md new file mode 100644 index 0000000..05b7553 --- /dev/null +++ b/ffsampledsp-x86_64-macos/CLAUDE.md @@ -0,0 +1,32 @@ +# ffsampledsp-x86_64-macos + +Canonical C source module for macOS x86_64. **All C sources live here.** Every other platform module (aarch64-macos, x86_64-linux, aarch64-linux, x86_64-win, i386-win) points its compiler at `../ffsampledsp-x86_64-macos/src/main/c/` — there is only one copy of the C code. + +## Build + +```bash +mvn --activate-profiles ffsampledsp-x86_64-macos install + +# Enable C-level debug output to stdout: +mvn --activate-profiles ffsampledsp-x86_64-macos install -Dcflags=-DDEBUG +``` + +Requires Apple Command Line Tools or Xcode. + +## C Sources (`src/main/c/`) + +| File | Role | +|---|---| +| `FFUtils.c` / `FFUtils.h` | Shared helpers: JNI field/method ID caching, direct `ByteBuffer` management, FFmpeg context lifecycle (`ff_init_audioio`, `ff_audioio_free`), DRM detection (`CODEC_TAG_DRMS`). Minimum probe score 5 avoids misdetection. | +| `FFAudioFileReader.c` | Implements `getAudioFileFormatsFromURL` and `getAudioFileFormatsFromBuffer`. Probes format with `avformat_open_input`; fills Java `FFAudioFileFormat` / `FFAudioFormat` objects via JNI. | +| `FFURLInputStream.c` | Opens `AVFormatContext` from a URL. `io_buffer_size` is set from the Java-side `fileBufferSize` argument. Decodes packets into the Java `nativeBuffer` (direct `ByteBuffer`). | +| `FFStreamInputStream.c` | Creates a custom `AVIOContext` with read callbacks that pull data from a Java `InputStream`. | +| `FFCodecInputStream.c` | Sets up `libswresample` for PCM conversion. Output sample format is derived from the Java `AudioFormat`: `AV_SAMPLE_FMT_S16` (16-bit signed), `AV_SAMPLE_FMT_FLT` (32-bit float), `AV_SAMPLE_FMT_DBL` (64-bit float), etc. Uses `av_opt_set_*()` API; `swr_alloc_set_opts()` is not used. | + +## JNI Headers + +Generated by `javac -h` during the `compile` phase of `ffsampledsp-java` into `../ffsampledsp-java/target/native/include/`. The native build consumes them from there — do not edit headers by hand. + +## Editing C Code + +Changes to any `.c`/`.h` file here affect **all platforms**. Always verify with at least one platform build before considering a change complete. diff --git a/ffsampledsp-x86_64-macos/src/main/c/FFAudioFileReader.c b/ffsampledsp-x86_64-macos/src/main/c/FFAudioFileReader.c index c6bc0be..be7d0a7 100644 --- a/ffsampledsp-x86_64-macos/src/main/c/FFAudioFileReader.c +++ b/ffsampledsp-x86_64-macos/src/main/c/FFAudioFileReader.c @@ -423,7 +423,7 @@ static int create_ffaudiofileformats(JNIEnv *env, */ JNIEXPORT jobjectArray JNICALL Java_com_tagtraum_ffsampledsp_FFAudioFileReader_getAudioFileFormatsFromURL( - JNIEnv *env, jobject instance, jstring url) { + JNIEnv *env, jobject instance, jstring url, jbyteArray urlBytes) { #ifdef DEBUG fprintf(stderr, "openFromUrl_1\n"); @@ -432,19 +432,23 @@ Java_com_tagtraum_ffsampledsp_FFAudioFileReader_getAudioFileFormatsFromURL( int res = 0; AVFormatContext *format_context = NULL; jobjectArray array = NULL; + char *input_url = NULL; init_ids(env); - // Use ff_jstring_to_utf8 instead of GetStringUTFChars: the JNI function - // returns Modified UTF-8 (CESU-8), which encodes supplementary characters - // (U+10000+, e.g. emoji) as 6 bytes rather than the standard UTF-8 4-byte - // sequence. File systems use standard UTF-8, so avformat_open_input would - // fail to find such files. - char *input_url = ff_jstring_to_utf8(env, url); - if (!input_url) { - throwIOExceptionIfError(env, AVERROR(ENOMEM), - "Failed to convert URL to UTF-8"); - goto bail; + { + jsize urlLen = (*env)->GetArrayLength(env, urlBytes); + jbyte *urlBuf = (*env)->GetByteArrayElements(env, urlBytes, NULL); + input_url = (char *)malloc(urlLen + 1); + if (!input_url) { + (*env)->ReleaseByteArrayElements(env, urlBytes, urlBuf, JNI_ABORT); + throwIOExceptionIfError(env, AVERROR(ENOMEM), + "Failed to allocate URL buffer"); + goto bail; + } + memcpy(input_url, urlBuf, urlLen); + input_url[urlLen] = '\0'; + (*env)->ReleaseByteArrayElements(env, urlBytes, urlBuf, JNI_ABORT); } res = ff_open_format_context(env, &format_context, input_url, 0); diff --git a/ffsampledsp-x86_64-macos/src/main/c/FFURLInputStream.c b/ffsampledsp-x86_64-macos/src/main/c/FFURLInputStream.c index a007faf..eb3e563 100644 --- a/ffsampledsp-x86_64-macos/src/main/c/FFURLInputStream.c +++ b/ffsampledsp-x86_64-macos/src/main/c/FFURLInputStream.c @@ -55,22 +55,26 @@ Java_com_tagtraum_ffsampledsp_FFURLInputStream_fillNativeBuffer( * @return pointer to new FFAudioIO */ JNIEXPORT jlong JNICALL Java_com_tagtraum_ffsampledsp_FFURLInputStream_open( - JNIEnv *env, jobject stream, jstring url, jint streamIndex, - jint fileBufferSize) { + JNIEnv *env, jobject stream, jstring url, jbyteArray urlBytes, + jint streamIndex, jint fileBufferSize) { int res = 0; FFAudioIO *aio = NULL; - - // Use ff_jstring_to_utf8 instead of GetStringUTFChars: the JNI function - // returns Modified UTF-8 (CESU-8), which encodes supplementary characters - // (U+10000+, e.g. emoji) as 6 bytes rather than the standard UTF-8 4-byte - // sequence. File systems use standard UTF-8, so avformat_open_input would - // fail to find such files. - char *input_url = ff_jstring_to_utf8(env, url); - if (!input_url) { - res = AVERROR(ENOMEM); - throwIOExceptionIfError(env, res, "Failed to convert URL to UTF-8"); - goto bail; + char *input_url = NULL; + + { + jsize urlLen = (*env)->GetArrayLength(env, urlBytes); + jbyte *urlBuf = (*env)->GetByteArrayElements(env, urlBytes, NULL); + input_url = (char *)malloc(urlLen + 1); + if (!input_url) { + (*env)->ReleaseByteArrayElements(env, urlBytes, urlBuf, JNI_ABORT); + res = AVERROR(ENOMEM); + throwIOExceptionIfError(env, res, "Failed to allocate URL buffer"); + goto bail; + } + memcpy(input_url, urlBuf, urlLen); + input_url[urlLen] = '\0'; + (*env)->ReleaseByteArrayElements(env, urlBytes, urlBuf, JNI_ABORT); } aio = calloc(1, sizeof(FFAudioIO)); diff --git a/ffsampledsp-x86_64-macos/src/main/c/FFUtils.c b/ffsampledsp-x86_64-macos/src/main/c/FFUtils.c index 9a41c53..eb2147d 100644 --- a/ffsampledsp-x86_64-macos/src/main/c/FFUtils.c +++ b/ffsampledsp-x86_64-macos/src/main/c/FFUtils.c @@ -1174,49 +1174,3 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM *vm, void *reserved) { avformat_network_init(); return JNI_VERSION_1_6; } - -char *ff_jstring_to_utf8(JNIEnv *env, jstring java_string) { - jclass string_class = NULL; - jmethodID get_bytes_mid = NULL; - jstring utf8_charset = NULL; - jbyteArray bytes = NULL; - jsize len = 0; - char *result = NULL; - - if (!java_string) - return NULL; - - string_class = (*env)->GetObjectClass(env, java_string); - if (!string_class) - goto bail; - - get_bytes_mid = (*env)->GetMethodID(env, string_class, "getBytes", - "(Ljava/lang/String;)[B"); - if (!get_bytes_mid) - goto bail; - - utf8_charset = (*env)->NewStringUTF(env, "UTF-8"); - if (!utf8_charset) - goto bail; - - bytes = (jbyteArray)(*env)->CallObjectMethod(env, java_string, get_bytes_mid, - utf8_charset); - if (!bytes) - goto bail; - - len = (*env)->GetArrayLength(env, bytes); - result = (char *)malloc(len + 1); - if (!result) - goto bail; - - (*env)->GetByteArrayRegion(env, bytes, 0, len, (jbyte *)result); - result[len] = '\0'; - -bail: - if (utf8_charset) - (*env)->DeleteLocalRef(env, utf8_charset); - if (bytes) - (*env)->DeleteLocalRef(env, bytes); - - return result; -} \ No newline at end of file diff --git a/ffsampledsp-x86_64-macos/src/main/c/FFUtils.h b/ffsampledsp-x86_64-macos/src/main/c/FFUtils.h index 0b78eb5..d1dc00e 100644 --- a/ffsampledsp-x86_64-macos/src/main/c/FFUtils.h +++ b/ffsampledsp-x86_64-macos/src/main/c/FFUtils.h @@ -116,25 +116,3 @@ AVCodec *ff_find_encoder(enum AVSampleFormat, int, int, int); int ff_init_encoder(JNIEnv *, FFAudioIO *, AVCodec *); int ff_big_endian(enum AVCodecID); - -/** - * Converts a Java string to a standard UTF-8 C string. - * - * JNI's GetStringUTFChars returns Modified UTF-8 (CESU-8), which encodes - * supplementary characters (U+10000 and above, e.g. emoji) as two 3-byte - * sequences (6 bytes total) instead of the single 4-byte sequence used by - * standard UTF-8. File systems on macOS and Linux use standard UTF-8 for file - * names, so paths containing emoji passed through GetStringUTFChars will not - * match the file on disk. - * - * This function calls java.lang.String.getBytes("UTF-8") via JNI, which always - * returns standard UTF-8 bytes regardless of the characters involved. - * - * The caller must free() the returned buffer when no longer needed. - * - * @param env JNI environment - * @param java_string Java String object - * @return Newly allocated null-terminated standard UTF-8 C string, - * or NULL on error - */ -char *ff_jstring_to_utf8(JNIEnv *env, jstring java_string); diff --git a/ffsampledsp-x86_64-win/CLAUDE.md b/ffsampledsp-x86_64-win/CLAUDE.md new file mode 100644 index 0000000..46cb305 --- /dev/null +++ b/ffsampledsp-x86_64-win/CLAUDE.md @@ -0,0 +1,17 @@ +# ffsampledsp-x86_64-win + +Native library module for Windows x86_64. Packages `ffsampledsp-x86_64-win.dll`. + +**No local C sources.** The compiler is pointed at `../ffsampledsp-x86_64-macos/src/main/c/` — edit C code there. + +## Build + +```bash +# Requires MSYS2 with MinGW-w64 GCC toolchain: +mvn --activate-profiles ffsampledsp-x86_64-win install + +# Debug build: +mvn --activate-profiles ffsampledsp-x86_64-win install -Dcflags=-DDEBUG +``` + +Windows URL handling note: `file:` URLs must follow the libav style (`file:C:/path/file`, not `file:///C:/path/file`). UNC paths use `file://server/path`. This conversion is done in `FFAudioFileReader.urlToString()`. diff --git a/src/site/apt/index.apt.vm b/src/site/apt/index.apt.vm index 5574938..8f41264 100644 --- a/src/site/apt/index.apt.vm +++ b/src/site/apt/index.apt.vm @@ -43,9 +43,11 @@ Introduction * Requirements - On macOS, requires OS X/macOS 10.10 or later and an Intel processor. Starting with v0.9.29, only 64 bit is supported. + requires Java 8 or later. + + On macOS, requires macOS 10.10 or later on Intel or Apple Silicon (aarch64). Only 64 bit is supported. On Windows, should run with Windows XP or higher. Both 32 bit and 64 bit are supported. - On Linux/Debian, should run with Ubuntu 20 or higher (x86/arm64). Only 64 bit is supported. + On Linux/Debian, should run with Ubuntu 20 or higher (x86_64/arm64). Only 64 bit is supported. * Stems Support @@ -59,6 +61,34 @@ Introduction For a code sample please see {{{./stems.html}here}}. +* File I/O Buffer Size + + When opening a <<>> or <<>> source, allocates an internal FFmpeg read + buffer (<<>>). The default size depends on the URL scheme: + + * For <<>> URLs: <<1 MB>>. This keeps system-call overhead low when reading large audio + files from disk. The default can be overridden at the JVM level with + <<<-Dffsampledsp.fileBufferSize=N>>>. + + * For non-<<>> URLs (e.g. HTTP/HTTPS): <<64 KB>>. The smaller default keeps latency low + when streaming over a network. The default can be overridden with + <<<-Dffsampledsp.urlBufferSize=N>>>. + + [] + + Both defaults can also be overridden per stream using the three-argument overloads of + <<>>: + ++-------------------------------+ +// Open stream 0 with an explicit 512 KB I/O buffer: +AudioInputStream in = new FFAudioFileReader() + .getAudioInputStream(file, 0, 512 * 1024); ++-------------------------------+ + + Larger buffers improve throughput for bulk transcoding or analysis; smaller buffers reduce + per-stream memory use and latency when streaming. The scheme-based defaults are a sensible + starting point for most applications. + * Alternatives For a 32/64 bit service provider implementation for Windows 7 or later, based on Microsoft's Media Foundation, @@ -78,11 +108,12 @@ Introduction There are numerous other Java-FFmpeg projects. Here's the short list, of the projects I'm aware of: - * {{{https://www.xuggle.com}Xuggle}} - Java wrapper around FFmpeg using JNI. + * {{{https://github.com/artclarke/xuggle-xuggler}Xuggler}} - Java wrapper around FFmpeg using JNI. Deprecated. * {{{https://github.com/MisterRager/jjmpeg}jjmpeg}} - thin Java wrapper around FFmpeg using JNI. * {{{https://www.sauronsoftware.it/projects/jave/}JAVE}} - Java wrapper around FFmpeg executables. + Apparently abandoned in 2012. * {{{http://fmj-sf.net/ffmpeg-java/getting_started.php}FFMPEG-Java}} - Java wrapper around FFmpeg, using {{{https://github.com/twall/jna}JNA}}. Apparently abandoned in 2007.