API definition¶

This API definition is based on the JavaScript Object Notation (JSON) Data Interchange Format as described by the Internet Engineering Task Force rfc8259. It is designed as an External OPeration code modules (XOP) for Igor Pro.

Concept¶

This XOP is based on nlohmann’s JSON for modern C++ parsing library. The principal concept is shown in the following image:

The conversion of Igor Pro Types to JSON Types requires some mappings that are defined in the following sections:

Types¶

Igor Pro does not provide direct mappings for all specified JSON Types. Most importantly, Igor Pro is not an object-oriented programming language and does not support objects directly. On the other hand, not all Igor Pro Types are available in the JSON representation either.

The API defines mappings for the conversion between available JSON Types and Igor Pro Types.

Exporting Igor Pro data to a JSON String (“dumping”) and Re-Importing the same data (“parsing”) yields identical data. In other words, parsing and dumping are bijective functions or a one-to-one correspondence regarding the export.

exporting data from Igor Pro to JSON is a bijective operation.¶

           export        re-import
**********  ---->  ******   ---->   **********
*Igor Pro*  <--->  *JSON*   <--->   *Igor Pro*
**********         ******           **********

Since Igor Pro and JSON types do not have equivalent representations, importing JSON strings (parse) from foreign instances like from python generated JSON and re-exporting (dump) them to JSON strings do not necessarily yield identical strings.

JSON Strings can be imported only if they match the rfc8259 specifications. Although, the parser will accept any JSON string.¶

       import             export
******  ---->  **********  ---->  ******
*JSON*  <-/->  *Igor Pro*  <--->  *JSON*
******         **********         ******

The following chapters summarize the available types for JSON and Igor Pro and define their conversion.

JSON Types¶

The rfc8259 standard differentiates between primitive and structured types:

JSON can represent four primitive types (strings, numbers, booleans, and null) and two structured types (objects and arrays). rfc8259

This API translates to the following JSON Types:

literal names
- true
- false
- null
primitive types
- strings
- numbers
- booleans
- null
structured types
- objects
- arrays

Igor Pro does not support the specified literal names that JSON requires. As a consequence, booleans and null types cannot be imported directly and need a mapping to the corresponding Igor Pro Types. More importantly, there is no direct representation in Igor Pro for the structured types: objects and arrays.

JSON types are referred to within Igor Pro using the following constants:

group JSON_TYPES

Variables

const double JSON_INVALID = -1¶

const double JSON_OBJECT = 0¶

const double JSON_ARRAY = 1¶

const double JSON_NUMERIC = 2¶

const double JSON_STRING = 3¶

const double JSON_BOOL = 4¶

const double JSON_NULL = 5¶

JSON Synchronisation Options¶

The synchronisation options define how objects a transferred from a source json to a target json. On synchronisation the json is traversed and each object is compared between source and target. The default constant JSON_SYNC_ADD_TO_TARGET only adds objects in the target if they are present in the source and not in the target. With the constant JSON_SYNC_REMOVE_IN_TARGET objects in the target are removed if they are not present in the source. With JSON_SYNC_OVERWRITE_IN_TARGET objects in the target are overwritten if they are present in the source and the target and both objects are different. All synchronisation options can be bitwise ORed. Note that from ORing all options target equals source follows.

group JSON_SYNC_OPTIONS

Variables

const double JSON_SYNC_ADD_TO_TARGET = 0x0¶

const double JSON_SYNC_REMOVE_IN_TARGET = 0x1¶

const double JSON_SYNC_OVERWRITE_IN_TARGET = 0x2¶

Igor Pro Types¶

Some Igor Pro Types like STRUCT and DFREF or complex variables and hexadecimal numbers (to name a few) do not have direct JSON representations. As a consequence, not all Igor Pro types will be available for export within this API, and some types need a translation.

The following Igor Pro Types are available to the API:

literal names
- NaN
- [+|-]inf
primitive types
- variables
- strings
structured types
- waves

Since the dump can only be achieved for a valid JSON object, the consequence is that the source data must be appropriately shaped to comply with the rfc8259 JSON specification before the export. If you want to preserve unusual variable content like for example complex variables you must define your own string serialization as by default they will be exported with their real part only.

On the contrary, the parser will accept all input that comes from any (non rfc8259 compliant) JSON representation and reads its content according to the following mappings:

Mappings¶

Since we do not have a one-to-one representation, the following mappings apply to convert between the Igor Pro and JSON representation:

Strings¶

A JSON representation of a string is a sequence of utf8 characters included in quotation marks. Quotation marks in the context of this XOP are double quotation marks “ (U+0022). In general, characters in JSON strings have to be escaped using the same escape character as in Igor Pro: the reverse solidus \ (U+005C). According to rfc8259 specification, the characters that must be escaped inside a JSON string with a reverse solidus are the following :

control characters: U+0000 through U+001F
quotation mark
reverse solidus: U+005C

DisplayHelpTopic "Escape Sequences in Strings"

Starting with Igor Pro 7, unicode is used internally. Also, JSON strings are utf8 based. In an attempt to handle one problem at a time, this XOP supports only Igor Pro in version 7 and higher. An implementation or transformation between different text encodings is out of the scope of this XOP.

DisplayHelpTopic "String Variables and Text Encodings"

Control Characters¶

Utilizing the unicode capabilities of Igor Pro, control characters like “r” (U+000D), “n” (U+000A) and “t” (U+0009) are saved directly in their utf8 representations but have to be escaped. Identically, any definitions of unicode characters like u0000 are saved as one character.

rfc8259 also requires some unusual non-printed characters to be escaped:

solidus U+002F
backspace U+0008
form feed U+000C

In the context of this XOP, specific care must be taken to escape any solidus within the name of object members inside JSON strings as the forward slash is also used to reference child-objects. See json path notation.

JSON representation¶

{
  "escaped\/subgroup": {
    "entity0": 0
  },
  "unescaped": {
    "subgroup": {
      "entity1": 1
    }
  }
}

Quotation Mark¶

Igor Pro only supports strings in double quotation marks. This allows us to build a one-to-one correspondence since JSON also uses utf8 based double-quoted strings.

In Igor Pro, the user escapes double quotation marks """ with an escape character. The string is then internally stored as (unescaped) unicode character. For the JSON representation, this quotation mark has to be escaped again.

String unicode, escape

escape = "\""
sprintf unicode, "U+%.4X", char2num(escape[0])
print !cmpstr(unicode, "U+0022") // --> "

escape = "'"
sprintf unicode, "U+%.4X", char2num(escape[0])
print !cmpstr(unicode, "U+0027") // --> '

Reverse Solidus¶

For a simple translation from Igor Pro into JSON strings, any remaining reverse solidus that is not handled by the rules defined in quotation mark and control characters is escaped using a reverse solidus. This also requires that upon conversion of a JSON string into an Igor Pro representation, any escaped characters have to be unescaped.

To the user, the Igor Pro string is left untouched on subsequent export and imports.

Single escaped specialties like the 3-digit octal byte representations for the white space character 040 are transformed to unicode within Igor Pro. To prevent any of these transformations, the solidus has to be escaped once within Igor so that a single solidus is stored in the string. The escape character is then recreated when it is stored in the JSON formatted output string.

Numbers¶

JSON is a pure ASCII based format. Therefore, it does not differentiate between different memory allocations of variables like the INT64 or Double Precision type. Since Igor Pro defaults to Single Precision for floating point numbers and INT32 for integers, any import from JSON to Igor Pro will reflect these two basic data types.

floating point¶

The differentiation between integers and floating point data types is done using the decimal separator ".". When storing integers, there will be no trailing fraction separator:

valid numbers in the JSON representation¶

{
  "integer": 42,
  "single0": 42.1,
  "single1": 4.2E1
}

On export, the data type can be forced to a specific type using the Adds a value to the JSON text: Operation.

Reducing the precision upon storing JSON objects is controllable. The JSON value then follows scientific precision notation with an exponential suffix.

JSON representation of a value with 4 digit precision.¶

{
  "integer": 4.200E1
}

special numbers¶

JSON assumes decimal numbers and does not support output to binary or hexadecimal numbers. You can use strings to define custom exports in Igor Pro.

possible JSON string for numbers with hexadecimal, binary and character representations¶

{
  "int": 42,
  "hex": "0x2A",
  "bin": "101010",
  "char": "*"
}

NaN and Inf¶

Special Igor Pro literal names like NaN and Inf are not supported as numbers in the JSON format. They are stored as strings and can be read with the Reads value(s) from the given location in the JSON text: operation bijectively as variables or optionally as strings.

JSON representation of a NaN¶

{
  "number": "NaN"
}

Although other parsers like json.dumps in python allow exporting these values as JSON numbers without double quotes (NaN and Inf), this behavior does not comply with rfc8259. By default, such numbers are considered invalid and treated as unspecified numbers. In fact, some JSON processors like jq will convert unquoted NaN and Inf to unexpected values as we are leaving the rfc specification. For this and other reasons, within this API we strictly stay rfc8259 compatible.

unspecified numbers¶

All numbers not matched by any of the above rules are imported as if its JSON value was null.

Booleans¶

Igor Pro does not know about booleans. They are transformed upon import from JSON using the following map:

true → 1

false → 0

null → NaN

Exporting to booleans can get forced by converting a value val with the following reverse operation:

!!numtype(val) → null

!!val → true

!val → false

null¶

A JSON null is converted to Igor Pro 0 for integer, float, and double read-outs, and to "" for string readout. If you want bijective operations, you should handle the readout of these types using JSON_GetType and your own definition. A convenient extension to RFC JSON is the bijective conversion of NaN, and Inf to string representations "NaN", and "Inf".

JSON path¶

Any entity in a JSON object can be identified with a JSON Pointer according to rfc6901. The specification generally follows POSIX path notation. But has some special notations i.e. for escaping solidus usage in names.

In the context of this framework, all paths are treated as absolute. The path prefix can, therefore, be omitted.