API definition

This API definition is based on the JavaScript Object Notation (JSON) Data Interchange Format as described by the Internet Engineering Task Force rfc8259. It is designed as an External OPeration code modules (XOP) for Igor Pro

Concept

This XOP is based on nlohmann’s JSON for modern C++ parsing library. The principal concept is shown in the following image:

_images/concept.svg

The conversion of Igor Pro Types to JSON Types requires some mappings that are defined in the following sections:

Types

Igor Pro does not provide direct mappings for all specified JSON Types. Most importantly, Igor Pro is not an object-oriented programming language and does not support objects directly. On the other hand, not all Igor Pro Types are available in the JSON representation either.

The API defines mappings for the conversion between available JSON Types and Igor Pro Types.

Exporting Igor Pro data to a JSON String (“dumping”) and Re-Importing the same data (“parsing”) yields identical data. In other words, parsing and dumping are bijective functions or a one-to-one correspondence regarding the export.

Since Igor Pro and JSON types do not have equivalent representations, importing JSON strings (parse) from foreign instances like from python generated JSON and re-exporting (dump) them to JSON strings do not necessarily yield identical strings.

The following chapters summarize the available types for JSON and Igor Pro and define their conversion.

JSON Types

The rfc8259 standard differentiates between primitive and structured types:

JSON can represent four primitive types (strings, numbers, booleans, and null) and two structured types (objects and arrays). rfc8259

This API translates to the following JSON Types:

Igor Pro does not support the specified literal names that JSON requires. As a consequence, booleans and null types cannot be imported directly and need a mapping to the corresponding Igor Pro Types. More importantly, there is no direct representation in Igor Pro for the structured types: objects and arrays.

JSON types are referred to within Igor Pro using the following constants:

group JSON_TYPES

Variables

const double JSON_INVALID = -1
const double JSON_OBJECT = 0
const double JSON_ARRAY = 1
const double JSON_NUMERIC = 2
const double JSON_STRING = 3
const double JSON_BOOL = 4
const double JSON_NULL = 5

Igor Pro Types

Some Igor Pro Types like STRUCT and DFREF or complex variables and hexadecimal numbers (to name a few) do not have direct JSON representations. As a consequence, not all Igor Pro types will be available for export within this API, and some types need a translation.

The following Igor Pro Types are available to the API:

Since the dump can only be achieved for a valid JSON object, the consequence is that the source data must be appropriately shaped to comply with the rfc8259 JSON specification before the export. If you want to preserve unusual variable content like for example complex variables you must define your own string serialization as by default they will be exported with their real part only.

On the contrary, the parser will accept all input that comes from any (non rfc8259 compliant) JSON representation and reads its content according to the following mappings:

Mappings

Since we do not have a one-to-one representation, the following mappings apply to convert between the Igor Pro and JSON representation:

Strings

A JSON representation of a string is a sequence of utf8 characters included in quotation marks. Quotation marks in the context of this XOP are double quotation marks (U+0022). In general, characters in JSON strings have to be escaped using the same escape character as in Igor Pro: the reverse solidus \ (U+005C). According to rfc8259 specification, the characters that must be escaped inside a JSON string with a reverse solidus are the following :

DisplayHelpTopic "Escape Sequences in Strings"

Starting with Igor Pro 7, unicode is used internally. Also, JSON strings are utf8 based. In an attempt to handle one problem at a time, this XOP supports only Igor Pro in version 7 and higher. An implementation or transformation between different text encodings is out of the scope of this XOP.

DisplayHelpTopic "String Variables and Text Encodings"

Control Characters

Utilizing the unicode capabilities of Igor Pro, control characters like “r” (U+000D), “n” (U+000A) and “t” (U+0009) are saved directly in their utf8 representations but have to be escaped. Identically, any definitions of unicode characters like \u0000 are saved as one character.

rfc8259 also requires some unusual non-printed characters to be escaped:

  • solidus U+002F
  • backspace U+0008
  • form feed U+000C

In the context of this XOP, specific care must be taken to escape any solidus within the name of object members inside JSON strings as the forward slash is also used to reference child-objects. See json path notation.

JSON representation
{
  "escaped\/subgroup": {
    "entity0": 0
  },
  "unescaped": {
    "subgroup": {
      "entity1": 1
    }
  }
}

Quotation Mark

Igor Pro only supports strings in double quotation marks. This allows us to build a one-to-one correspondence since JSON also uses utf8 based double-quoted strings.

In Igor Pro, the user escapes double quotation marks "\"" with an escape character. The string is then internally stored as (unescaped) unicode character. For the JSON representation, this quotation mark has to be escaped again.

String unicode, escape

escape = "\""
sprintf unicode, "U+%.4X", char2num(escape[0])
print !cmpstr(unicode, "U+0022") // --> "

escape = "'"
sprintf unicode, "U+%.4X", char2num(escape[0])
print !cmpstr(unicode, "U+0027") // --> '

Reverse Solidus

For a simple translation from Igor Pro into JSON strings, any remaining reverse solidus that is not handled by the rules defined in quotation mark and control characters is escaped using a reverse solidus. This also requires that upon conversion of a JSON string into an Igor Pro representation, any escaped characters have to be unescaped.

To the user, the Igor Pro string is left untouched on subsequent export and imports.

Single escaped specialties like the 3-digit octal byte representations for the white space character \040 are transformed to unicode within Igor Pro. To prevent any of these transformations, the solidus has to be escaped once within Igor so that a single solidus is stored in the string. The escape character is then recreated when it is stored in the JSON formatted output string.

Numbers

JSON is a pure ASCII based format. Therefore, it does not differentiate between different memory allocations of variables like the INT64 or Double Precision type. Since Igor Pro defaults to Single Precision for floating point numbers and INT32 for integers, any import from JSON to Igor Pro will reflect these two basic data types.

floating point

The differentiation between integers and floating point data types is done using the decimal separator ".". When storing integers, there will be no trailing fraction separator:

valid numbers in the JSON representation
{
  "integer": 42,
  "single0": 42.1,
  "single1": 4.2E1
}

On export, the data type can be forced to a specific type using the add a value Operation.

Reducing the precision upon storing JSON objects is controllable. The JSON value then follows scientific precision notation with an exponential suffix.

JSON representation of a value with 4 digit precision.
{
  "integer": 4.200E1
}

special numbers

JSON assumes decimal numbers and does not support output to binary or hexadecimal numbers. You can use strings to define custom exports in Igor Pro.

possible JSON string for numbers with hexadecimal, binary and character representations
{
  "int": 42,
  "hex": "0x2A",
  "bin": "101010",
  "char": "*"
}

NaN and Inf

Special Igor Pro literal names like NaN and Inf are not supported as numbers in the JSON format. They are stored as strings and can be read with the get a value operation bijectively as variables or optionally as strings.

JSON representation of a NaN
{
  "number": "NaN"
}

Although other parsers like json.dumps in python allow exporting these values as JSON numbers without double quotes (NaN and Inf), this behavior does not comply with rfc8259. By default, such numbers are considered invalid and treated as unspecified numbers. In fact, some JSON processors like jq will convert unquoted NaN and Inf to unexpected values as we are leaving the rfc specification. For this and other reasons, within this API we strictly stay rfc8259 compatible.

unspecified numbers

All numbers not matched by any of the above rules are imported as if its JSON value was null.

Booleans

Igor Pro does not know about booleans. They are transformed upon import from JSON using the following map:

  • true1
  • false0
  • nullNaN

Exporting to booleans can get forced by converting a value val with the following reverse operation:

  • !!numtype(val)null
  • !!valtrue
  • !valfalse

null

Any unusual number that has no representation in JSON is mapped to null. A JSON null is converted to Igor Pro NaN. If you want bijective operations, you should handle unsupported numbers as strings as it explained in special numbers section.

  • !!numtype(val)null
  • nullNaN

JSON path

Any entity in a JSON object can be identified with a JSON Pointer according to rfc6901. The specification generally follows POSIX path notation. But has some special notations i.e. for escaping solidus usage in names.

In the context of this framework, all paths are treated as absolute. The path prefix can, therefore, be omitted.