ritz/RESEARCH.md

# Ritz Framework Research & Architecture

## Core Insight: Python-Executable, WASM-Compilable

The breakthrough: Our framework uses **valid Python syntax** that can execute in a Python interpreter for debugging/prototyping, but compiles to WebAssembly for production. Developers write Python, debug in Python, ship as WASM.

## The Yield-Based Component Model

### Pure Python Implementation (for prototyping/debugging)

```python
# This is ACTUAL Python that runs in CPython!
from ritz import component, signal, Main, P, Label, Button

@component
def HighFiveApp():
    high_fives = signal(0)
    
    def increment():
        high_fives.value += 1
    
    with Main():
        yield P('Give me a high 5 bro!')
        yield Label(f'High fives: {high_fives}')
        yield Button('Click me', onclick=increment)

# Can execute in Python for debugging:
# python app.py -> generates HTML/IR for verification
```

### Complex Example with Mixed Content

```python
@component
def BlogPost(title, author, content):
    with Article(class_='blog-post'):
        with Header():
            yield H1(title)
            yield 'By '                    # Raw strings!
            yield Span(author, class_='author')
            yield ' on '
            yield Time(content.date)
        
        with Section():
            yield from content.paragraphs   # Composition via yield from
            
            for comment in content.comments:
                yield Comment(comment.author, comment.text)
```

### Why This Architecture Works

1. **Debugging**: Run components in Python interpreter to see generated HTML/IR
2. **Type Checking**: Use Python's type hints for compile-time validation
3. **Testing**: Standard Python test frameworks work
4. **Compilation**: Parser only needs to handle subset of Python, not full language

## Compilation Pipeline

### 1. Parser (Python → AST) - Using Python's `ast` Module!

**BREAKTHROUGH**: Since our components are valid Python, we can use Python's own `ast` module to parse them! No custom parser needed!

```python
import ast
import inspect

def parse_component(func):
    """Parse a component function using Python's AST"""
    source = inspect.getsource(func)
    tree = ast.parse(source)
    
    # Walk the AST and transform to our IR
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef):
            # Found our component function
            if has_component_decorator(node):
                return transform_to_ritz_ast(node)

def transform_to_ritz_ast(node):
    """Transform Python AST to Ritz IR"""
    component = {
        'type': 'component',
        'name': node.name,
        'body': []
    }
    
    for stmt in node.body:
        if isinstance(stmt, ast.With):
            # with Element(): ...
            component['body'].append(process_with_statement(stmt))
        elif isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Yield):
            # yield Element()
            component['body'].append(process_yield(stmt.value))
    
    return component
```

This gives us:
- **Zero parsing overhead** - Python does it for us
- **Perfect Python compatibility** - It IS Python
- **Error messages for free** - Python's parser handles syntax errors
- **Type hints support** - Can extract from AST

### 2. LLVM IR Generation

```cpp
// codegen.cpp
class RitzCodeGen {
    llvm::LLVMContext context;
    llvm::IRBuilder<> builder;
    llvm::Module* module;
    
    // Compile @component function to WASM
    llvm::Function* compileComponent(ASTNode* node) {
        auto func = llvm::Function::Create(
            llvm::FunctionType::get(builder.getVoidTy(), false),
            llvm::Function::ExternalLinkage,
            node->component.name,
            module
        );
        
        auto entry = llvm::BasicBlock::Create(context, "entry", func);
        builder.SetInsertPoint(entry);
        
        // Process yields - each becomes DOM operations
        for (auto stmt : node->component.body) {
            if (stmt->type == AST_YIELD) {
                compileYield(stmt);
            }
        }
        
        builder.CreateRetVoid();
        return func;
    }
    
    void compileYield(ASTNode* node) {
        // yield P('text') compiles to:
        // 1. Create P element
        // 2. Create text node
        // 3. Append text to P
        // 4. Append P to current parent
        
        auto create_p = builder.CreateCall(
            module->getFunction("dom_create_element"),
            builder.getInt32(TAG_P)
        );
        
        // ... rest of DOM operations
    }
};
```

### 3. Reactive Runtime (in WASM)

```c
// runtime.c - Compiles to WASM
typedef struct {
    int32_t value;
    uint32_t subscribers[MAX_SUBSCRIBERS];
    uint32_t subscriber_count;
    uint32_t dom_node_id;  // Direct DOM node reference
} Signal;

Signal* create_signal(int32_t initial) {
    Signal* sig = (Signal*)wasm_alloc(sizeof(Signal));
    sig->value = initial;
    sig->subscriber_count = 0;
    return sig;
}

void signal_set(Signal* sig, int32_t value) {
    sig->value = value;
    
    // Direct DOM update - no virtual DOM!
    if (sig->dom_node_id) {
        dom_ops[op_count++] = OP_UPDATE_TEXT;
        dom_ops[op_count++] = sig->dom_node_id;
        dom_ops[op_count++] = (uint32_t)int_to_string(value);
    }
    
    // Notify subscribers
    for (int i = 0; i < sig->subscriber_count; i++) {
        call_effect(sig->subscribers[i]);
    }
}
```

## DOM Bridge Architecture

### WASM Side - Accumulate Operations

```c
// Opcodes for DOM operations
enum DOMOp {
    OP_CREATE_ELEMENT = 0x01,
    OP_CREATE_TEXT = 0x02,
    OP_SET_ATTRIBUTE = 0x03,
    OP_APPEND_CHILD = 0x04,
    OP_UPDATE_TEXT = 0x05,
    OP_ADD_EVENT = 0x06,
    OP_REMOVE = 0x07,
    OP_BATCH_END = 0xFF
};

// Global operation buffer
uint32_t dom_ops[MAX_OPS];
uint32_t op_count = 0;

// Batch operations then flush
void flush_dom_ops() {
    // Call JS with pointer to operation buffer
    js_flush_dom_ops(dom_ops, op_count);
    op_count = 0;
}
```

### JavaScript Side - Execute Operations

```javascript
// bridge.js
const nodeMap = new Map();
let nodeIdCounter = 1;

function flushDOMOps(ptr, len) {
    const ops = new Uint32Array(wasmMemory.buffer, ptr, len);
    const stack = [];
    
    for (let i = 0; i < len;) {
        switch(ops[i++]) {
            case 0x01: { // CREATE_ELEMENT
                const tagId = ops[i++];
                const el = document.createElement(TAG_NAMES[tagId]);
                const id = nodeIdCounter++;
                nodeMap.set(id, el);
                stack.push(id);
                break;
            }
            
            case 0x05: { // UPDATE_TEXT
                const nodeId = ops[i++];
                const textPtr = ops[i++];
                const text = readString(textPtr);
                nodeMap.get(nodeId).textContent = text;
                break;
            }
            
            // ... other operations
        }
    }
}
```

## Implementation Roadmap

### Phase 1: Python Prototype (Week 1)
- [x] Define component syntax with yield
- [ ] Build Python runtime for testing (framework.py)
- [ ] Create HTML generation from component tree
- [ ] Signal/reactive state in pure Python

### Phase 2: Parser (Week 2)
- [ ] Python subset grammar definition
- [ ] Parser using ANTLR or hand-written recursive descent
- [ ] AST generation from Python code
- [ ] Semantic analysis (type checking, reactive dependency tracking)

### Phase 3: WASM Compiler (Week 3-4)  
- [ ] LLVM setup and IR generation
- [ ] Compile components to WASM functions
- [ ] Reactive runtime in C/Rust
- [ ] DOM operation batching

### Phase 4: JavaScript Bridge (Week 5)
- [ ] DOM operation interpreter in JS
- [ ] Event handler callbacks
- [ ] String passing optimization
- [ ] Memory management

### Phase 5: Optimization (Week 6)
- [ ] Template compilation (static vs dynamic parts)
- [ ] Signal-to-DOM direct mapping
- [ ] Batch operation compression
- [ ] Bundle size optimization

## Key Technical Decisions

### Why Yield?
- Natural Python syntax for building trees
- Composable via `yield from`
- Clear execution order
- No magic - just generators

### Why Compile vs Interpret?
- 10-100x performance improvement
- 5-10x smaller bundle (no Python runtime)
- Direct DOM updates (no virtual DOM)
- Type safety at compile time

### Why LLVM?
- Mature toolchain
- Excellent WASM backend
- Optimization passes
- Debugging support

## Performance Targets

Based on research of SolidJS, Leptos, and Dioxus:

- **Bundle size**: < 200KB compressed
- **Startup time**: < 500ms on slow 3G  
- **Update latency**: Within 10% of vanilla JS
- **Memory usage**: 70% less than React
- **DOM mutations**: 99% fewer than virtual DOM

## Critical Implementation Details

### Signal Subscription Model
```c
// Each signal tracks its subscribers
// Each effect tracks its dependencies
// Bidirectional relationship for cleanup

typedef struct Effect {
    void (*fn)(void);
    Signal** dependencies;
    uint32_t dep_count;
} Effect;

// Global context during effect execution
Effect* current_effect = NULL;

// When signal.get() called, register dependency
int32_t signal_get(Signal* sig) {
    if (current_effect) {
        add_dependency(current_effect, sig);
    }
    return sig->value;
}
```

### Memory Management
- Bump allocator for per-frame allocations
- Generation-based GC for long-lived objects
- Reference counting for shared resources
- No Rc/RefCell exposed to users

### String Handling
- String table in WASM memory
- Pass indices, not pointers
- Intern common strings
- UTF-8 throughout

## Revolutionary Debugging Story

### Python Debugging in WASM! 

**THE INSIGHT**: We're not debugging JavaScript - we're debugging Python code running in WASM! This means we can:

```python
# This is what you see in DevTools
@component
def Counter():
    count = signal(0)  # <-- Breakpoint here shows Python source!
    
    def increment():
        count += 1     # <-- Step through Python, not JS!
        print(f"Count: {count}")  # <-- See Python values
```

### How It Works

1. **DWARF Debug Symbols**: LLVM generates standard DWARF symbols in WASM
2. **Source Maps on Steroids**: Map WASM instructions → Python source lines
3. **Symbol Translation**: 
   ```
   WASM offset 0x1234 → Counter.py:15:8 → "count += 1"
   ```
4. **Browser Integration**: Chrome DevTools already supports WASM debugging!
5. **Live Editing**: Recompile just the changed component, hot-reload the WASM module

### Debugging Options

```javascript
// In browser console
wasm.debug.setBreakpoint('Counter.py', 15)
wasm.debug.stepInto()
wasm.debug.inspect('count')  // Shows Signal { value: 5, subscribers: [...] }

// Or even connect Python debugger!
# pdb remote debugging over WebSocket
import pdb; pdb.set_trace()  # This could actually work!
```

## Modular Loading Architecture

### Progressive WASM Bundles

Instead of one monolithic WASM file, split into modules:

```
bootstrap.wasm (50KB)
├── Minimal runtime
├── Login component
└── Module loader

app.wasm (200KB) - Loaded after auth
├── Main application
├── Core components
└── Router

lazy/
├── admin.wasm (150KB) - Admin panel
├── analytics.wasm (100KB) - Analytics dashboard
├── reports.wasm (120KB) - Report generation
└── [feature].wasm - Feature-specific modules
```

### Security Through Architecture

```python
# bootstrap.py - Compiles to bootstrap.wasm
@component
def LoginPage():
    with Main():
        yield LoginForm()
        # No app code here - not even downloaded!

# After successful auth
async def load_app():
    # Server only sends this to authenticated users
    app_module = await fetch('/wasm/app.wasm', {
        'headers': {'Authorization': f'Bearer {token}'}
    })
    wasm.load_module(app_module)
```

### Implementation

```javascript
// module-loader.js
class RitzModuleLoader {
    constructor() {
        this.modules = new Map();
        this.cache = new Cache('ritz-modules-v1');
    }
    
    async loadModule(name, url) {
        // Check cache first
        const cached = await this.cache.match(url);
        if (cached) {
            const wasm = await WebAssembly.instantiate(cached);
            this.modules.set(name, wasm.instance);
            return;
        }
        
        // Download and cache
        const response = await fetch(url);
        const bytes = await response.arrayBuffer();
        
        // Cache for next time
        await this.cache.put(url, new Response(bytes));
        
        // Instantiate
        const wasm = await WebAssembly.instantiate(bytes, {
            env: this.getImports(name)
        });
        
        this.modules.set(name, wasm.instance);
    }
}
```

### Route-Based Code Splitting

```python
# router.py
@component
def Router():
    current_route = signal(window.location.pathname)
    
    with Main():
        if current_route == '/admin':
            # Lazy load admin module
            yield LazyLoad('admin.wasm', AdminPanel)
        elif current_route == '/reports':
            # Only load if needed
            yield LazyLoad('reports.wasm', ReportsView)
        else:
            # Main app is already loaded
            yield Dashboard()
```

## Next Steps (Updated)

1. ✅ **Prototype**: Build framework.py with working signals and components
2. **AST Parser**: Use Python's `ast` module to parse components
3. **Compiler Pipeline**: 
   - Python AST → Ritz IR
   - Ritz IR → LLVM IR
   - LLVM IR → WASM with debug symbols
4. **Debugging Infrastructure**:
   - Generate source maps
   - Embed Python source in WASM
   - Browser DevTools integration
5. **Module System**:
   - Build tool for splitting modules
   - Runtime module loader
   - Caching strategy
6. **Test**: Port TodoMVC with lazy-loaded features

## References

- SolidJS: Fine-grained reactivity without virtual DOM
- Leptos: Rust signals with Copy + 'static  
- Dioxus Sledgehammer: Batched DOM operations
- LLVM WebAssembly Backend: https://llvm.org/docs/WebAssemblyUsage.html
- Binaryen: WASM optimizer and code generator

## Core Innovation

**Python syntax → WASM performance**. Write and debug in Python, ship as WASM. No virtual DOM, direct signal-to-DOM updates, 99% fewer mutations than React. The future of web frameworks: compiled, reactive, Python-like.