ritz/RESEARCH.md

Ritz Framework Research & Architecture

Core Insight: Python-Executable, WASM-Compilable

The breakthrough: Our framework uses valid Python syntax that can execute in a Python interpreter for debugging/prototyping, but compiles to WebAssembly for production. Developers write Python, debug in Python, ship as WASM.

The Yield-Based Component Model

Pure Python Implementation (for prototyping/debugging)

# This is ACTUAL Python that runs in CPython!
from ritz import component, signal, Main, P, Label, Button

@component
def HighFiveApp():
    high_fives = signal(0)
    
    def increment():
        high_fives.value += 1
    
    with Main():
        yield P('Give me a high 5 bro!')
        yield Label(f'High fives: {high_fives}')
        yield Button('Click me', onclick=increment)

# Can execute in Python for debugging:
# python app.py -> generates HTML/IR for verification

Complex Example with Mixed Content

@component
def BlogPost(title, author, content):
    with Article(class_='blog-post'):
        with Header():
            yield H1(title)
            yield 'By '                    # Raw strings!
            yield Span(author, class_='author')
            yield ' on '
            yield Time(content.date)
        
        with Section():
            yield from content.paragraphs   # Composition via yield from
            
            for comment in content.comments:
                yield Comment(comment.author, comment.text)

Why This Architecture Works

1. Debugging: Run components in Python interpreter to see generated HTML/IR
2. Type Checking: Use Python's type hints for compile-time validation
3. Testing: Standard Python test frameworks work
4. Compilation: Parser only needs to handle subset of Python, not full language

Compilation Pipeline

1. Parser (Python → AST) - Using Python's ast Module!

BREAKTHROUGH: Since our components are valid Python, we can use Python's own ast module to parse them! No custom parser needed!

import ast
import inspect

def parse_component(func):
    """Parse a component function using Python's AST"""
    source = inspect.getsource(func)
    tree = ast.parse(source)
    
    # Walk the AST and transform to our IR
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef):
            # Found our component function
            if has_component_decorator(node):
                return transform_to_ritz_ast(node)

def transform_to_ritz_ast(node):
    """Transform Python AST to Ritz IR"""
    component = {
        'type': 'component',
        'name': node.name,
        'body': []
    }
    
    for stmt in node.body:
        if isinstance(stmt, ast.With):
            # with Element(): ...
            component['body'].append(process_with_statement(stmt))
        elif isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Yield):
            # yield Element()
            component['body'].append(process_yield(stmt.value))
    
    return component

This gives us:

• Zero parsing overhead - Python does it for us
• Perfect Python compatibility - It IS Python
• Error messages for free - Python's parser handles syntax errors
• Type hints support - Can extract from AST

2. LLVM IR Generation

// codegen.cpp
class RitzCodeGen {
    llvm::LLVMContext context;
    llvm::IRBuilder<> builder;
    llvm::Module* module;
    
    // Compile @component function to WASM
    llvm::Function* compileComponent(ASTNode* node) {
        auto func = llvm::Function::Create(
            llvm::FunctionType::get(builder.getVoidTy(), false),
            llvm::Function::ExternalLinkage,
            node->component.name,
            module
        );
        
        auto entry = llvm::BasicBlock::Create(context, "entry", func);
        builder.SetInsertPoint(entry);
        
        // Process yields - each becomes DOM operations
        for (auto stmt : node->component.body) {
            if (stmt->type == AST_YIELD) {
                compileYield(stmt);
            }
        }
        
        builder.CreateRetVoid();
        return func;
    }
    
    void compileYield(ASTNode* node) {
        // yield P('text') compiles to:
        // 1. Create P element
        // 2. Create text node
        // 3. Append text to P
        // 4. Append P to current parent
        
        auto create_p = builder.CreateCall(
            module->getFunction("dom_create_element"),
            builder.getInt32(TAG_P)
        );
        
        // ... rest of DOM operations
    }
};

3. Reactive Runtime (in WASM)

// runtime.c - Compiles to WASM
typedef struct {
    int32_t value;
    uint32_t subscribers[MAX_SUBSCRIBERS];
    uint32_t subscriber_count;
    uint32_t dom_node_id;  // Direct DOM node reference
} Signal;

Signal* create_signal(int32_t initial) {
    Signal* sig = (Signal*)wasm_alloc(sizeof(Signal));
    sig->value = initial;
    sig->subscriber_count = 0;
    return sig;
}

void signal_set(Signal* sig, int32_t value) {
    sig->value = value;
    
    // Direct DOM update - no virtual DOM!
    if (sig->dom_node_id) {
        dom_ops[op_count++] = OP_UPDATE_TEXT;
        dom_ops[op_count++] = sig->dom_node_id;
        dom_ops[op_count++] = (uint32_t)int_to_string(value);
    }
    
    // Notify subscribers
    for (int i = 0; i < sig->subscriber_count; i++) {
        call_effect(sig->subscribers[i]);
    }
}

DOM Bridge Architecture

WASM Side - Accumulate Operations

// Opcodes for DOM operations
enum DOMOp {
    OP_CREATE_ELEMENT = 0x01,
    OP_CREATE_TEXT = 0x02,
    OP_SET_ATTRIBUTE = 0x03,
    OP_APPEND_CHILD = 0x04,
    OP_UPDATE_TEXT = 0x05,
    OP_ADD_EVENT = 0x06,
    OP_REMOVE = 0x07,
    OP_BATCH_END = 0xFF
};

// Global operation buffer
uint32_t dom_ops[MAX_OPS];
uint32_t op_count = 0;

// Batch operations then flush
void flush_dom_ops() {
    // Call JS with pointer to operation buffer
    js_flush_dom_ops(dom_ops, op_count);
    op_count = 0;
}

JavaScript Side - Execute Operations

// bridge.js
const nodeMap = new Map();
let nodeIdCounter = 1;

function flushDOMOps(ptr, len) {
    const ops = new Uint32Array(wasmMemory.buffer, ptr, len);
    const stack = [];
    
    for (let i = 0; i < len;) {
        switch(ops[i++]) {
            case 0x01: { // CREATE_ELEMENT
                const tagId = ops[i++];
                const el = document.createElement(TAG_NAMES[tagId]);
                const id = nodeIdCounter++;
                nodeMap.set(id, el);
                stack.push(id);
                break;
            }
            
            case 0x05: { // UPDATE_TEXT
                const nodeId = ops[i++];
                const textPtr = ops[i++];
                const text = readString(textPtr);
                nodeMap.get(nodeId).textContent = text;
                break;
            }
            
            // ... other operations
        }
    }
}

Implementation Roadmap

Phase 1: Python Prototype (Week 1)

• Define component syntax with yield
• Build Python runtime for testing (framework.py)
• Create HTML generation from component tree
• Signal/reactive state in pure Python

Phase 2: Parser (Week 2)

• Python subset grammar definition
• Parser using ANTLR or hand-written recursive descent
• AST generation from Python code
• Semantic analysis (type checking, reactive dependency tracking)

Phase 3: WASM Compiler (Week 3-4)

• LLVM setup and IR generation
• Compile components to WASM functions
• Reactive runtime in C/Rust
• DOM operation batching

Phase 4: JavaScript Bridge (Week 5)

• DOM operation interpreter in JS
• Event handler callbacks
• String passing optimization
• Memory management

Phase 5: Optimization (Week 6)

• Template compilation (static vs dynamic parts)
• Signal-to-DOM direct mapping
• Batch operation compression
• Bundle size optimization

Key Technical Decisions

Why Yield?

• Natural Python syntax for building trees
• Composable via yield from
• Clear execution order
• No magic - just generators

Why Compile vs Interpret?

• 10-100x performance improvement
• 5-10x smaller bundle (no Python runtime)
• Direct DOM updates (no virtual DOM)
• Type safety at compile time

Why LLVM?

• Mature toolchain
• Excellent WASM backend
• Optimization passes
• Debugging support

Performance Targets

Based on research of SolidJS, Leptos, and Dioxus:

• Bundle size: < 200KB compressed
• Startup time: < 500ms on slow 3G
• Update latency: Within 10% of vanilla JS
• Memory usage: 70% less than React
• DOM mutations: 99% fewer than virtual DOM

Critical Implementation Details

Signal Subscription Model

// Each signal tracks its subscribers
// Each effect tracks its dependencies
// Bidirectional relationship for cleanup

typedef struct Effect {
    void (*fn)(void);
    Signal** dependencies;
    uint32_t dep_count;
} Effect;

// Global context during effect execution
Effect* current_effect = NULL;

// When signal.get() called, register dependency
int32_t signal_get(Signal* sig) {
    if (current_effect) {
        add_dependency(current_effect, sig);
    }
    return sig->value;
}

Memory Management

• Bump allocator for per-frame allocations
• Generation-based GC for long-lived objects
• Reference counting for shared resources
• No Rc/RefCell exposed to users

String Handling

• String table in WASM memory
• Pass indices, not pointers
• Intern common strings
• UTF-8 throughout

Revolutionary Debugging Story

Python Debugging in WASM!

THE INSIGHT: We're not debugging JavaScript - we're debugging Python code running in WASM! This means we can:

# This is what you see in DevTools
@component
def Counter():
    count = signal(0)  # <-- Breakpoint here shows Python source!
    
    def increment():
        count += 1     # <-- Step through Python, not JS!
        print(f"Count: {count}")  # <-- See Python values

How It Works

1. DWARF Debug Symbols: LLVM generates standard DWARF symbols in WASM
2. Source Maps on Steroids: Map WASM instructions → Python source lines
3. Symbol Translation:

   WASM offset 0x1234 → Counter.py:15:8 → "count += 1"
   

4. Browser Integration: Chrome DevTools already supports WASM debugging!
5. Live Editing: Recompile just the changed component, hot-reload the WASM module

Debugging Options

// In browser console
wasm.debug.setBreakpoint('Counter.py', 15)
wasm.debug.stepInto()
wasm.debug.inspect('count')  // Shows Signal { value: 5, subscribers: [...] }

// Or even connect Python debugger!
# pdb remote debugging over WebSocket
import pdb; pdb.set_trace()  # This could actually work!

Modular Loading Architecture

Progressive WASM Bundles

Instead of one monolithic WASM file, split into modules:

bootstrap.wasm (50KB)
├── Minimal runtime
├── Login component
└── Module loader

app.wasm (200KB) - Loaded after auth
├── Main application
├── Core components
└── Router

lazy/
├── admin.wasm (150KB) - Admin panel
├── analytics.wasm (100KB) - Analytics dashboard
├── reports.wasm (120KB) - Report generation
└── [feature].wasm - Feature-specific modules

Security Through Architecture

# bootstrap.py - Compiles to bootstrap.wasm
@component
def LoginPage():
    with Main():
        yield LoginForm()
        # No app code here - not even downloaded!

# After successful auth
async def load_app():
    # Server only sends this to authenticated users
    app_module = await fetch('/wasm/app.wasm', {
        'headers': {'Authorization': f'Bearer {token}'}
    })
    wasm.load_module(app_module)

Implementation

// module-loader.js
class RitzModuleLoader {
    constructor() {
        this.modules = new Map();
        this.cache = new Cache('ritz-modules-v1');
    }
    
    async loadModule(name, url) {
        // Check cache first
        const cached = await this.cache.match(url);
        if (cached) {
            const wasm = await WebAssembly.instantiate(cached);
            this.modules.set(name, wasm.instance);
            return;
        }
        
        // Download and cache
        const response = await fetch(url);
        const bytes = await response.arrayBuffer();
        
        // Cache for next time
        await this.cache.put(url, new Response(bytes));
        
        // Instantiate
        const wasm = await WebAssembly.instantiate(bytes, {
            env: this.getImports(name)
        });
        
        this.modules.set(name, wasm.instance);
    }
}

Route-Based Code Splitting

# router.py
@component
def Router():
    current_route = signal(window.location.pathname)
    
    with Main():
        if current_route == '/admin':
            # Lazy load admin module
            yield LazyLoad('admin.wasm', AdminPanel)
        elif current_route == '/reports':
            # Only load if needed
            yield LazyLoad('reports.wasm', ReportsView)
        else:
            # Main app is already loaded
            yield Dashboard()

Next Steps (Updated)

1. ✅ Prototype: Build framework.py with working signals and components
2. AST Parser: Use Python's ast module to parse components
3. Compiler Pipeline:
   • Python AST → Ritz IR
   • Ritz IR → LLVM IR
   • LLVM IR → WASM with debug symbols
4. Debugging Infrastructure:
   • Generate source maps
   • Embed Python source in WASM
   • Browser DevTools integration
5. Module System:
   • Build tool for splitting modules
   • Runtime module loader
   • Caching strategy
6. Test: Port TodoMVC with lazy-loaded features

References

• SolidJS: Fine-grained reactivity without virtual DOM
• Leptos: Rust signals with Copy + 'static
• Dioxus Sledgehammer: Batched DOM operations
• LLVM WebAssembly Backend: https://llvm.org/docs/WebAssemblyUsage.html
• Binaryen: WASM optimizer and code generator

Core Innovation

Python syntax → WASM performance. Write and debug in Python, ship as WASM. No virtual DOM, direct signal-to-DOM updates, 99% fewer mutations than React. The future of web frameworks: compiled, reactive, Python-like.