Description

A minimal, token-efficient data format optimized for LLMs and Agentic AI workflows.

Author: Mahesh Vaikri

—

ISON (Interchange Simple Object Notation) is a minimal data interchange format optimized for Large Language Models.
It is easy for humans to read and write. It is easy for LLMs to understand and generate.
It is based on familiar tabular and relational patterns that language models have seen billions of times in training data.

ISON is a text format that is completely language independent but represents data in a way that maximizes token efficiency
and minimizes cognitive load for AI systems. These properties make ISON an ideal data interchange format for AI and LLM workflows.

—

Why ISON?

ISON reduces token usage by 30-70% compared to JSON while remaining human-readable and LLM-friendly.

JSON (87 tokens)                    ISON (34 tokens)
─────────────────                   ─────────────────
{                                   table.users
  "users": [                        id:int name:string email active:bool
    {                               1 Alice alice@example.com true
      "id": 1,                      2 Bob bob@example.com false
      "name": "Alice",              3 Charlie charlie@example.com true
      "email": "alice@example.com",
      "active": true
    },
    {
      "id": 2,
      "name": "Bob",
      "email": "bob@example.com",
      "active": false
    },
    {
      "id": 3,
      "name": "Charlie",
      "email": "charlie@example.com",
      "active": true
    }
  ]
}

Perfect for:

• Multi-agent systems
• RAG pipelines
• Graph databases
• Token-constrained AI/ML, LLM, Agentic AI workflows
• LLM function calling

—

Quick Start

Installation

JavaScript/TypeScript:

npm install ison-parser    # JavaScript
npm install ison-ts        # TypeScript with full types
npm install isonantic-ts   # Validation & schemas

Python:

pip install ison-py        # Parser
pip install isonantic      # Validation & schemas

Rust:

[dependencies]
ison-rs = "1.0"
isonantic-rs = "1.0"       # Validation & schemas

C++ (Header-only):

Just copy the header
cp ison-cpp/include/ison_parser.hpp /your/project/

Go:

go get github.com/maheshvaikri-code/ison/ison-go
go get github.com/maheshvaikri-code/ison/isonantic-go

Usage Examples

JavaScript:

import { parse, dumps, toJSON } from 'ison-parser';
const doc = parse(
table.users
id:int name:string active:bool
1 Alice true
2 Bob false
);
console.log(doc.users.rows);
// [{ id: 1, name: 'Alice', active: true }, ...]
console.log(toJSON(doc));
// Standard JSON output

Python:

from isonpy import parse, dumps, tojson
doc = parse("""
table.users
id:int name:string active:bool
1 Alice true
2 Bob false
""")
for row in doc['users']['rows']:
    print(f"{row['id']}: {row['name']}")
Convert to JSON
print(to_json(doc))

Rust:

use ison_rs::{parse, dumps};
let doc = parse(r#"
table.users
id:int name:string active:bool
1 Alice true
2 Bob false
"#)?;
let users = doc.get("users").unwrap();
for row in &users.rows {
    let name = row.get("name").andthen(|v| v.asstr()).unwrap();
    println!("{}", name);
}

C++:

#include "ison_parser.hpp"
auto doc = ison::parse(R"(
table.users
id:int name:string active:bool
1 Alice true
2 Bob false
)");
for (const auto& row : doc["users"].rows) {
    std::cout << ison::as_string(row.at("name")) << std::endl;
}

Go:

import "github.com/maheshvaikri-code/ison/ison-go"
doc, _ := ison.Parse(
table.users
id:int name:string active:bool
1 Alice true
2 Bob false
)
users, _ := doc.Get("users")
for _, row := range users.Rows {
    name, _ := row["name"].AsString()
    fmt.Println(name)
}

---

ISON Format

Comments start with #
table.users                        # Block: kind.name
id:int name:string email active:bool   # Fields with optional types
1 Alice alice@example.com true     # Data rows (space-separated)
2 "Bob Smith" bob@example.com false    # Quoted strings for spaces
3 ~ ~ true                         # ~ or null for null values
table.orders
id user_id product
1 :1 Widget                        # :1 = reference to id 1
2 :user:42 Gadget                  # :user:42 = namespaced reference
object.config                      # Single-row object block
key value
debug true
---                                # Summary separator
count 100                          # Summary row

ISONL (Streaming Format)

For large datasets, use line-based ISONL where each line is self-contained:

table.users|id name email|1 Alice alice@example.com
table.users|id name email|2 Bob bob@example.com

---

Packages

| Ecosystem | Parser | Validation | Status |
|-----------|--------|------------|--------|
| NPM | ison-parser | isonantic-ts | 33 + 46 tests |
| NPM | ison-ts | - | 23 tests |
| PyPI | ison-py | isonantic | 31 + 39 tests |
| Crates.io | ison-rs | isonantic-rs | 9 + 1 tests |
| C++ | ison-cpp | isonantic-cpp | 30 tests |
| Go | ison-go | isonantic-go | 28 + 55 tests |

Total: 11 packages across 5 ecosystems, 303+ tests passing

---

Features

| Feature | Description |
|---------|-------------|
| Tables | Structured data with typed columns |
| Objects | Single-row key-value blocks |
| References | :id, :type:id, :RELATIONSHIP:id |
| Type Annotations | field:int, field:string, field:bool, field:float |
| Computed Fields | field:computed for derived values |
| ISONL Streaming | Line-based format for large datasets |
| JSON Export | Convert ISON to JSON |
| Roundtrip | Parse and serialize without data loss |

---

Schema Validation (ISONantic)

Type-safe validation with fluent API:

// JavaScript/TypeScript
import { table, string, int, boolean } from 'isonantic-ts';
const userSchema = table('users')
  .field('id', int().required())
  .field('name', string().min(1).max(100))
  .field('email', string().email())
  .field('active', boolean().default(true));
const users = userSchema.validate(doc);

Python
from isonantic import table, string, int_, boolean
user_schema = (table('users')
    .field('id', int_().required())
    .field('name', string().min(1).max(100))
    .field('email', string().email())
    .field('active', boolean().default(True)))
users = user_schema.validate(doc)

---

Documentation

• Website: www.ison.dev
• Getting Started: www.getison.com
• Specification: ISON v1.0 Spec
• Playground: Interactive Demo

---

Project Structure

ison/
├── ison-js/               # JavaScript parser (NPM: ison-parser)
├── ison-ts/               # TypeScript parser (NPM: ison-ts)
├── isonantic-ts/          # TypeScript validation (NPM: isonantic-ts)
├── ison-py/               # Python parser (PyPI: ison-py)
├── isonantic/             # Python validation (PyPI: isonantic)
├── ison-rust/             # Rust parser (Crates.io: ison-rs)
├── isonantic-rust/        # Rust validation (Crates.io: isonantic-rs)
├── ison-cpp/              # C++ header-only parser
├── isonantic-cpp/         # C++ header-only validation
├── ison-go/               # Go parser
├── isonantic-go/          # Go validation
├── benchmark/             # Token efficiency benchmarks
├── images/                # Logo and assets
├── LICENSE                # MIT License
└── README.md              # This file

---

Development

Clone the repository
git clone https://github.com/maheshvaikri-code/ison.git
cd ison
JavaScript/TypeScript
cd ison-js && npm install && npm test
cd ison-ts && npm install && npm test
cd isonantic-ts && npm install && npm test
Python
cd ison-py && pip install -e . && pytest
cd isonantic && pip install -e . && pytest
Rust
cd ison-rust && cargo test
cd isonantic-rust && cargo test
C++
cd ison-cpp && mkdir build && cd build && cmake .. && cmake --build . && ctest
Go
cd ison-go && go test -v ./...
cd isonantic-go && go test -v ./...

---

Test Results

Click to expand test results (303+ tests passing)

JavaScript (ison-parser) - 33 tests

✓ parses basic table correctly
✓ handles quoted strings
✓ preserves type annotations
✓ handles references
✓ parses multiple tables
✓ converts to JSON correctly
✓ handles null values
✓ handles empty values in rows
✓ handles ISONL format

TypeScript (ison-ts) - 23 tests

✓ should parse basic table
✓ should handle quoted strings
✓ should preserve type annotations
✓ should handle references
✓ should parse multiple tables
✓ should convert to JSON
✓ should handle null values
✓ should handle empty values
✓ should parse ISONL format

TypeScript Validation (isonantic-ts) - 46 tests

✓ validates required string fields
✓ validates optional string fields
✓ validates string min/max length
✓ validates email format
✓ validates int/float/bool fields
✓ validates references
✓ validates table schemas
... and 39 more tests

Python (ison-py) - 31 tests

✓ testparsebasic_table
✓ testparsequoted_strings
✓ testparsetype_annotations
✓ testparsereferences
✓ testparsemultiple_tables
✓ testtojson
✓ testdumps / testdumps_isonl
... and 24 more tests

Python Validation (isonantic) - 39 tests

✓ teststringfield_required
✓ teststringmin_length
✓ testemailvalidation
✓ testintfield / testfloatfield
✓ testreferencefield
✓ testtableschema
... and 33 more tests

Rust (ison-rs) - 9 tests

✓ testdumpswith_delimiter
✓ test_isonl
✓ testisonto_json
✓ testjsonto_ison
✓ testparsereferences
✓ testparsesimple_table
✓ test_roundtrip
✓ testtypeinference
✓ test_version
✓ doc-tests

C++ (ison-cpp) - 30 tests

✓ parsesimpletable
✓ parseobjectblock
✓ parsemultipleblocks
✓ type_inference (int, float, bool, null, string)
✓ parse_references (simple, namespaced, relationship)
✓ serialize_roundtrip
✓ parseisonl / serializeisonl
✓ to_json
... and 15 more tests

Go (ison-go) - 28 tests

✓ TestParseSimpleTable
✓ TestParseTypedFields
✓ TestParseQuotedStrings
✓ TestParseNullValues
✓ TestParseReferences
✓ TestParseObjectBlock
✓ TestParseMultipleBlocks
✓ TestDumps / TestRoundtrip
✓ TestDumpsISONL / TestParseISONL
✓ TestToJSON / TestFromJSON
... and 18 more tests

Go Validation (isonantic-go) - 55 tests

✓ TestStringRequired / TestStringOptional
✓ TestStringMinLength / TestStringMaxLength
✓ TestStringEmail / TestStringURL
✓ TestNumberMin / TestNumberPositive
✓ TestIntSchema / TestBooleanRequired
✓ TestRefRequired / TestRefNamespace
✓ TestObjectFieldValidation
✓ TestTableRowValidation
✓ TestDocumentParse / TestDocumentSafeParse
... and 46 more tests

---

Benchmark Results 🏆

300-Question Benchmark across 20 datasets using GPT-4o tokenizer (o200k_base):

| Format | Total Tokens | vs JSON | Accuracy | Acc/1K Tokens |
|--------|-------------|---------|----------|---------------|
| ISON | 3,550 | -72.0% | 88.3% | 24.88 |
| TOON | 4,847 | -61.7% | 88.7% | 18.29 |
| JSON Compact | 7,339 | -42.1% | 89.0% | 12.13 |
| JSON | 12,668 | baseline | 84.7% | 6.68 |

Key Results

• ✅ ISON won ALL 20 token benchmarks
• ✅ 272% more efficient than JSON (Accuracy per 1K tokens)
• ✅ 27% more token-efficient than TOON
• ✅ 3.6x more data in same context window

👉 Full Benchmark Details | Run the Benchmark

---

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

License

MIT License - see LICENSE for details.

---

Author

Mahesh Vaikri

• Website: www.ison.dev | www.getison.com
• GitHub: @maheshvaikri-code

---

ISON - Less tokens, more context, better AI.