Skip to content

docs: improve README example #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
embray merged 11 commits into from
Jun 25, 2026
Merged

docs: improve README example #201

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c4870e9
docs: improve README example
embray Jun 22, 2026
02ef7ee
docs: ensure proper asdf_value_destroy calls in README example
embray Jun 22, 2026
a87d743
test: fix the automake docs examples wrapper script to log stderr
embray Jun 22, 2026
bde46d2
docs: improve readme with simpler "Getting started" examples
embray Jun 23, 2026
d32b3d8
build: fix missing documentation files from source tarball
embray Jun 23, 2026
16cfa07
docs: missing return statement in example
embray Jun 23, 2026
8935dbd
docs: changes to example file to avoid certain confusions
embray Jun 24, 2026
8392b1b
docs: use READE examples suggested by @braingram
embray Jun 25, 2026
5757d66
docs: update example output in readme to reflect gh-204 fixes
embray Jun 25, 2026
daa836d
test: the readme examples must still support taking a filename argument
embray Jun 25, 2026
14ca599
build: work around macOS endianness function declaration problems
embray Jun 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
259 changes: 163 additions & 96 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ This returns an ``asdf_file_t *`` which is your main interface to the ASDF file.
When done with the file make sure to call ``asdf_close`` to free resources:

.. code:: c
:name: test-open-close-file
:test: test-open-close-file
:fixture: cube.asdf

#include <stdio.h>
#include <asdf.h>
Expand All @@ -57,120 +58,186 @@ When done with the file make sure to call ``asdf_close`` to free resources:
return 0;
}

The following more complete example demonstrates how to read different metadata out of
the ASDF tree, as well as extract block data. Inline comments provide further explanation:
The next example demonstrates how to write some simple metadata and an ndarray
to a new file:

.. code:: c
:name: test-read-metadata-ndarray
:test: test-write-file

@braingram braingram Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting the example into 2 programs (asdf-write and asdf-read) improves the approach-ability of the example(s) and makes them easier to follow.

However, this produces an invalid file (the serialized time is invalid).

     'id': 'http://stsci.edu/schemas/asdf/time/time-1.4.0',
     'title': 'Represents an instance in time.'}

On instance:
    {'value': 'J1948.78707178', 'format': 'iso_time'}

The format also doesn't match roman files yet the example mentions Roman which is misleading. Roman files do not have any "obstime" or "observer" keys and the "data" is nested under a parent "roman" key.

Let's avoid linking libasdf to roman here in the README and cover that elsewhere (where roman files can be used).

I'd also say we avoid anything FITS-like here and cover that elsewhere.

How about an example file similar to the python example with a tree containing?:

  • name: Dennis Ritchie
  • foo: 42
  • sequence: array of numbers 1-99
  • powers/squares: squares of sequence

(leaving out random since it seems overly complicated for this example)

The write could cover:

  • setting a scalar string/number
  • creating an array
  • nesting data

Read could cover accessing all of these values (and perhaps summing the sequence and squares).

@embray embray Jun 24, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not so sure about any of this to be honest.

On instance:
{'value': 'J1948.78707178', 'format': 'iso_time'}

I don't know how you got that. Might be using an older version of the library? The generated file has a copy committed in the repository (for comparison in the tests). It clearly outputs:

obstime: !time/time-1.4.0
  value: J1948.78707178
  format: jyear

This is valid according to the schema (which I think itself is a bit buggy, but, the fact that this is accepted seems reasonable): https://github.com/asdf-format/asdf-standard/blob/main/resources/stable/schemas/stsci.edu/asdf/time/time-1.4.0.yaml

Though interestingly when I tried to open the file in Python I got a different exception -- it passed schema validation and threw:

ValueError: Input values did not match the format class jyear:
TypeError: for jyear class, input should be (long) doubles, string, or Decimal, and second values are only allowed for (long) doubles.

This looks like a bug through and through. Makes no sense.

I could try changing the example to a different time format for now just to avoid any confusion from this, but will investigate the upstream bug...

The format also doesn't match roman files yet the example mentions Roman which is misleading. Roman files do not have any "obstime" or "observer" keys and the "data" is nested under a parent "roman" key.

This statement is also a bit confusing to me. It's just using Nancy Roman as an example "observer". I guess her name is on my mind for obvious reasons. I don't think anyone is ever confused in FITS examples when "OBSERVER = Edwin Hubble" that the example represents a valid HST data product.

But sure I could change it to Dennis Ritchie or Daffy Duck for all it matters.

How about an example file similar to the python example with a tree containing?

I like the idea of exactly mirroring the Python example in principle. But unfortunately I don't think it translates as well to the C library in a succinct way. In Python, preparing a dictionary of small ndarrays is practically a one-liner. In C each one of those examples is several lines of setup and teardown code that has to be managed, bogging down the simplicity of the example.

The above statement is maybe also a good argument for adding some helper macros for creating simple ndarrays, or mapping builders. This is something I've been thinking about, but probably out of scope for now.

@braingram braingram Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not so sure about any of this to be honest.

On instance:
{'value': 'J1948.78707178', 'format': 'iso_time'}

I don't know how you got that. Might be using an older version of the library?

Likely yes. Pulling down fresh code and recompiling the library I don't see the odd 'iso_time'.

The generated file has a copy committed in the repository (for comparison in the tests). It clearly outputs:

obstime: !time/time-1.4.0
  value: J1948.78707178
  format: jyear

This is valid according to the schema (which I think itself is a bit buggy, but, the fact that this is accepted seems reasonable): https://github.com/asdf-format/asdf-standard/blob/main/resources/stable/schemas/stsci.edu/asdf/time/time-1.4.0.yaml

I wouldn't trust this schema to be a complete description of how time formats are handled. This is another reason to avoid using time in this example as that code has not been fully vetted.

This statement is also a bit confusing to me. It's just using Nancy Roman as an example "observer". I guess her name is on my mind for obvious reasons. I don't think anyone is ever confused in FITS examples when "OBSERVER = Edwin Hubble" that the example represents a valid HST data product.

But sure I could change it to Dennis Ritchie or Daffy Duck for all it matters.

How about an example file similar to the python example with a tree containing?

I like the idea of exactly mirroring the Python example in principle. But unfortunately I don't think it translates as well to the C library in a succinct way. In Python, preparing a dictionary of small ndarrays is practically a one-liner. In C each one of those examples is several lines of setup and teardown code that has to be managed, bogging down the simplicity of the example.

Here's the candidate examples with some inline comments to help readers associate API with specific operations:

#include <asdf.h>

int main(int argc, char **argv) {
    const char *filename = "out.asdf";

    // open a "NULL" file for writing
    asdf_file_t *file = asdf_open(NULL);

    // assign a string to the "name" key of the ASDF tree
    asdf_set_string0(file, "name", "Dennis Richie");

    // assign a numeric value to the "foo" key
    asdf_set_int64(file, "foo", 42);

    // construct 2 arrays containing numeric values
    uint64_t N = 100;

    asdf_ndarray_t sequence = {
        .ndim = 1,
        .shape = (uint64_t[]){N},
        .datatype = {.type = ASDF_DATATYPE_UINT64}
    };
    uint8_t *sequence_data = asdf_ndarray_data_alloc(&sequence);

    asdf_ndarray_t squares = {
        .ndim = 1,
        .shape = (uint64_t[]){N},
        .datatype = {.type = ASDF_DATATYPE_UINT64}
    };
    uint64_t *squares_data = asdf_ndarray_data_alloc(&squares);

    for (int idx = 0; idx < N; idx++) {
        sequence_data[idx] = idx;
        squares_data[idx] = idx * idx;
    };

    // assign the "sequence" array to the "sequence" key
    asdf_set_ndarray(file, "sequence", &sequence);

    // nest the "squares" array under a parent "powers" key
    asdf_set_ndarray(file, "powers/squares", &squares);

    // write the ASDF file to disk
    asdf_write_to(file, filename);

    // clean up allocations
    asdf_ndarray_data_dealloc(&sequence);
    asdf_ndarray_data_dealloc(&squares);
    asdf_close(file);
    return 0;
 }

and reading:

#include <stdio.h>
#include <stdlib.h>
#include <asdf.h>

int main(int argc, char **argv) {
    const char *filename = "out.asdf";

    // open the ASDF file for reading
    asdf_file_t *file = asdf_open(filename, "r");
    if (!file) {
        fprintf(stderr, "Failed to open the file: %s\n", asdf_error(file));
        return 1;
    }

    // read and print the string stored under "name"
    const char *name = NULL;
    if (asdf_get_string0(file, "name", &name) == ASDF_VALUE_OK) {
        printf("name: %s\n", name);
    }

    // read and print the numeric value stored under "foo"
    int64_t foo = 0;
    if (asdf_get_int64(file, "foo", &foo) == ASDF_VALUE_OK) {
        printf("foo: %lli\n", foo);
    }

    // read the "squares" array nested under the "powers" key
    asdf_ndarray_t *squares = NULL;
    uint64_t *squares_data = NULL;
    if (asdf_get_ndarray(file, "powers/squares", &squares) == ASDF_VALUE_OK) {
        if (asdf_ndarray_read_all(squares, ASDF_DATATYPE_UINT64, (void **)&squares_data) == ASDF_NDARRAY_OK) {
            // print the sum of the squares array
            uint64_t nelem = asdf_ndarray_size(squares);
            uint64_t sum = 0;
            for (uint64_t idx = 0; idx < nelem; idx++) {
                sum += squares_data[idx];
            }
            printf("sum of squares values: %lli\n", sum);
        }
    }

    // clean up allocations
    free(squares_data);
    asdf_ndarray_destroy(squares);
    asdf_close(file);
    return 0;
 }

I think it's helpful here to show:

  • read/writing string and numeric values to the tree
  • reading and writing arrays
  • that ASDF is hierarchical (by nesting the "squares" array under "powers")

@embray embray Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, fair enough, I'll use those.

:fixture: temp:out.asdf

#include <asdf.h>

int main(int argc, char **argv) {
const char *filename = "out.asdf";

if (argc > 1)
filename = argv[1];

// open a "NULL" file for writing
asdf_file_t *file = asdf_open(NULL);

// assign a string to the "name" key of the ASDF tree
asdf_set_string0(file, "name", "Dennis Richie");

// assign a numeric value to the "foo" key
asdf_set_int64(file, "foo", 42);

// construct 2 arrays containing numeric values
uint64_t N = 100;

asdf_ndarray_t sequence = {
.ndim = 1,
.shape = (uint64_t[]){N},
.datatype = {.type = ASDF_DATATYPE_UINT64}
};
uint8_t *sequence_data = asdf_ndarray_data_alloc(&sequence);

asdf_ndarray_t squares = {
.ndim = 1,
.shape = (uint64_t[]){N},
.datatype = {.type = ASDF_DATATYPE_UINT64}
};
uint64_t *squares_data = asdf_ndarray_data_alloc(&squares);

for (uint64_t idx = 0; idx < N; idx++) {
sequence_data[idx] = idx;
squares_data[idx] = idx * idx;
};

// assign the "sequence" array to the "sequence" key
asdf_set_ndarray(file, "sequence", &sequence);

// nest the "squares" array under a parent "powers" key
asdf_set_ndarray(file, "powers/squares", &squares);

// write the ASDF file to disk
asdf_write_to(file, filename);

// clean up allocations
asdf_ndarray_data_dealloc(&sequence);
asdf_ndarray_data_dealloc(&squares);
asdf_close(file);
return 0;
}

With libasdf installed on your system (see :ref:`development`) you can compile
and run this test like:

.. code:: sh

$ gcc asdf-write.c -o asdf-write -lasdf
$ ./asdf-write

It should produce an output file at ``out.asdf`` which you can inspect by hand.
The YAML portion of the ASDF file should contain:

.. code:: yaml

#ASDF 1.0.0
#ASDF_STANDARD 1.6.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
asdf_library: !core/software-1.0.0
name: libasdf
version: 0.1.0a2
author: The libasdf Developers
homepage: https://github.com/asdf-format/libasdf
name: Dennis Richie
foo: 42
sequence: !core/ndarray-1.1.0
source: 0
datatype: uint64
shape: [
100
]
byteorder: little
powers:
squares: !core/ndarray-1.1.0
source: 1
datatype: uint64
shape: [
100
]
byteorder: little
...


The next example shows how to read back in the same file:

.. code:: c
:test: test-read-file
:fixture: test-write-file.asdf

#include <stdio.h>
#include <stdlib.h>
#include <asdf.h>

int main(int argc, char **argv) {
if (argc < 2) {
fprintf(stderr, "Usage: %s filename\n", argv[0]);
return 1;
}
const char *filename = argv[1];
const char *filename = "out.asdf";

// The mode string "r" is required and is the only currently-supported mode
if (argc > 1)
filename = argv[1];

// open the ASDF file for reading
asdf_file_t *file = asdf_open(filename, "r");

if (file == NULL) {
fprintf(stderr, "error opening the ASDF file\n");
if (!file) {
fprintf(stderr, "Failed to open the file: %s\n", asdf_error(file));
return 1;
}

// The simplest way to read metadata from the file is with the
// `asdf_get_<type>*` family of functions
// They all return a value by pointer argument and return an
// `asdf_value_error_t`
// For example you can read a string from the metadata like:

const char *software = NULL;
// Returns a 0-terminated string into *software.
asdf_value_err_t err = asdf_get_string0(file, "asdf_library/author", &software);

if (err == ASDF_VALUE_OK) {
printf("software: %s\n", software);
}

// Other errors could be e.g. ASDF_VALUE_ERR_NOT_FOUND if the key doesn't
// exist, or ASDF_VALUE_ERR_TYPE_MISMATCH if it's not a string.

// There are also extensions registered for some (not all yet) of the
// core schemas. Objects defined by extension schemas (identified by
// their YAML tags) also have corresponding asdf_get_<type> functions:
asdf_meta_t *meta = NULL;

// This reads the top-level core/asdf-1.0.0 schema
err = asdf_get_meta(file, "/", &meta);
if (err == ASDF_VALUE_OK) {
if (meta->history.entries[0]) {
// This is a NULL-terminated array of asdf_history_entry_t*
printf("first history entry: %s\n", meta->history.entries[0]->description);
}
}

// Functions like `asdf_get_meta` that return into a double-pointer to a
// struct allocate memory for that structure automatically.
// The all have a corresponding `asdf_<type>_destroy` function.
// The plan is to track these on the file object (issue #34) to make
// memory management easier and cleaner, but for now you have to free
// them manually when you're done with them. This is good practice in any
// case.
asdf_meta_destroy(meta);

// ndarrays work no differently; this reads an ndarray named "cube".
asdf_ndarray_t *ndarray = NULL;
err = asdf_get_ndarray(file, "cube", &ndarray);
if (err != ASDF_VALUE_OK) {
fprintf(stderr, "error reading ndarray metadata: %d\n", err);
return 1;

// read and print the string stored under "name"
const char *name = NULL;
if (asdf_get_string0(file, "name", &name) == ASDF_VALUE_OK) {
printf("name: %s\n", name);
}

printf("number of data dimensions: %d\n", ndarray->ndim);

// Get just a raw pointer to the ndarray data block (if uncompressed).
// Optionally returns the size in bytes as well
size_t size = 0;
const void *data = asdf_ndarray_data(ndarray, &size);

if (data == NULL) {
fprintf(stderr, "error reading ndarray data\n");
return 1;

// read and print the numeric value stored under "foo"
int64_t foo = 0;
if (asdf_get_int64(file, "foo", &foo) == ASDF_VALUE_OK) {
printf("foo: %li\n", foo);
}

// Slightly more useful is the asdf_ndarray_read_tile_ functions.
// They can copy the data, including converting endianness into a tile
// buffer. If an existing buffer is not passed it will allocate one of
// the correct size to hold the data. The user is responsible for
// freeing the buffer.

// Read a 10x10x10 cube
const uint64_t origin[3] = {0, 0, 0};
const uint64_t shape[3] = {10, 10, 10};
void *tile = NULL;
asdf_ndarray_err_t array_err = asdf_ndarray_read_tile_ndim(
ndarray,
origin,
shape,
ASDF_DATATYPE_SOURCE,
&tile
);

if (array_err != ASDF_NDARRAY_OK) {
fprintf(stderr, "error reading ndarray: %d\n", array_err);
return 1;

// read the "squares" array nested under the "powers" key
asdf_ndarray_t *squares = NULL;
uint64_t *squares_data = NULL;
if (asdf_get_ndarray(file, "powers/squares", &squares) == ASDF_VALUE_OK) {
if (asdf_ndarray_read_all(squares, ASDF_DATATYPE_UINT64, (void **)&squares_data) == ASDF_NDARRAY_OK) {
// print the sum of the squares array
uint64_t nelem = asdf_ndarray_size(squares);
uint64_t sum = 0;
for (uint64_t idx = 0; idx < nelem; idx++) {
sum += squares_data[idx];
}
printf("sum of squares values: %li\n", sum);
}
}

free(tile);
asdf_ndarray_destroy(ndarray);

// clean up allocations
free(squares_data);
asdf_ndarray_destroy(squares);
asdf_close(file);
return 0;
}

Likewise compile and run the example with the output from the previous program:

.. code:: sh

$ gcc asdf-read.c -o asdf-read -lasdf
$ ./asdf-read

This should output::

name: Dennis Richie
foo: 42
sum of squares values: 328350

Additional examples can be found in the
`libasdf documentation <https://libasdf.readthedocs.io/en/latest/usage/examples.html>`__.


.. _development:

Development
===========
Expand Down
16 changes: 16 additions & 0 deletions docs/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,24 @@ CMAKE_DIST = CMakeLists.txt
EXTRA_DIST = \
$(CMAKE_DIST) \
conf.py \
api/asdf/core/datatype.h.rst \
api/asdf/core/ndarray.h.rst \
api/asdf/error.h.rst \
api/asdf/emitter.h.rst \
api/asdf/error.h.rst \
api/asdf/extension.h.rst \
api/asdf/file.h.rst \
api/asdf/value.h.rst \
api/asdf/yaml.h.rst \
environment.yml \
index.rst \
links.rst \
usage/examples.rst \
usage/extensions.rst \
usage/opening.rst \
usage/overview.rst \
usage/values.rst \
usage/writing.rst \
_static/css/globalnav.css \
_static/images/favicon.ico \
_static/images/logo-dark-mode.png \
Expand Down
22 changes: 22 additions & 0 deletions docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
from datetime import datetime
from pathlib import Path

from docutils.parsers.rst import directives
from sphinx.directives.patches import Code


# -- Project information ------------------------------------------------------
def read_config_h() -> tuple[str, str, str]:
Expand Down Expand Up @@ -169,5 +172,24 @@ def read_config_h() -> tuple[str, str, str]:
latex_logo = "_static/images/logo-light-mode.png"


# -- Doc-example test directive options ----------------------------------------
# The tests/scripts/extract_doc_examples.py script extracts ``.. code:: c``
# blocks from the documentation and compiles/runs them as part of the test
# suite. A block is marked for extraction with the ``:test:`` option (whose
# value is the test name) and may declare an input file with ``:fixture:``.
#
# These options are meaningful only to the extraction script; here we simply
# extend the ``code`` directive to accept (and otherwise ignore) them so that
# the documentation still builds without "unknown option" errors.
#
# TODO: Make this more extensible; maybe spin out to a separate plugin
# Sphinx could use an extension for compilable source code doctests like this...
class TestableCode(Code):
option_spec = dict(Code.option_spec)
option_spec["test"] = directives.unchanged
option_spec["fixture"] = directives.unchanged


def setup(app):
app.add_css_file("css/globalnav.css")
app.add_directive("code", TestableCode, override=True)
1 change: 1 addition & 0 deletions docs/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Usage
usage/values
usage/writing
usage/extensions
usage/examples


API documentation
Expand Down
Loading
Loading