Initial commit - some junk
This commit is contained in:
A2IA committed
commit
316c090778
9 files changed
+2272
No files matched your search
+522
@@ -0,0 +1,522 @@
|
||||
# Ritz Framework Research & Architecture
|
||||
|
||||
## Core Insight: Python-Executable, WASM-Compilable
|
||||
|
||||
The breakthrough: Our framework uses **valid Python syntax** that can execute in a Python interpreter for debugging/prototyping, but compiles to WebAssembly for production. Developers write Python, debug in Python, ship as WASM.
|
||||
|
||||
## The Yield-Based Component Model
|
||||
|
||||
### Pure Python Implementation (for prototyping/debugging)
|
||||
|
||||
```python
|
||||
# This is ACTUAL Python that runs in CPython!
|
||||
from ritz import component, signal, Main, P, Label, Button
|
||||
|
||||
@component
|
||||
def HighFiveApp():
|
||||
high_fives = signal(0)
|
||||
|
||||
def increment():
|
||||
high_fives.value += 1
|
||||
|
||||
with Main():
|
||||
yield P('Give me a high 5 bro!')
|
||||
yield Label(f'High fives: {high_fives}')
|
||||
yield Button('Click me', onclick=increment)
|
||||
|
||||
# Can execute in Python for debugging:
|
||||
# python app.py -> generates HTML/IR for verification
|
||||
```
|
||||
|
||||
### Complex Example with Mixed Content
|
||||
|
||||
```python
|
||||
@component
|
||||
def BlogPost(title, author, content):
|
||||
with Article(class_='blog-post'):
|
||||
with Header():
|
||||
yield H1(title)
|
||||
yield 'By ' # Raw strings!
|
||||
yield Span(author, class_='author')
|
||||
yield ' on '
|
||||
yield Time(content.date)
|
||||
|
||||
with Section():
|
||||
yield from content.paragraphs # Composition via yield from
|
||||
|
||||
for comment in content.comments:
|
||||
yield Comment(comment.author, comment.text)
|
||||
```
|
||||
|
||||
### Why This Architecture Works
|
||||
|
||||
1. **Debugging**: Run components in Python interpreter to see generated HTML/IR
|
||||
2. **Type Checking**: Use Python's type hints for compile-time validation
|
||||
3. **Testing**: Standard Python test frameworks work
|
||||
4. **Compilation**: Parser only needs to handle subset of Python, not full language
|
||||
|
||||
## Compilation Pipeline
|
||||
|
||||
### 1. Parser (Python → AST) - Using Python's `ast` Module!
|
||||
|
||||
**BREAKTHROUGH**: Since our components are valid Python, we can use Python's own `ast` module to parse them! No custom parser needed!
|
||||
|
||||
```python
|
||||
import ast
|
||||
import inspect
|
||||
|
||||
def parse_component(func):
|
||||
"""Parse a component function using Python's AST"""
|
||||
source = inspect.getsource(func)
|
||||
tree = ast.parse(source)
|
||||
|
||||
# Walk the AST and transform to our IR
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef):
|
||||
# Found our component function
|
||||
if has_component_decorator(node):
|
||||
return transform_to_ritz_ast(node)
|
||||
|
||||
def transform_to_ritz_ast(node):
|
||||
"""Transform Python AST to Ritz IR"""
|
||||
component = {
|
||||
'type': 'component',
|
||||
'name': node.name,
|
||||
'body': []
|
||||
}
|
||||
|
||||
for stmt in node.body:
|
||||
if isinstance(stmt, ast.With):
|
||||
# with Element(): ...
|
||||
component['body'].append(process_with_statement(stmt))
|
||||
elif isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Yield):
|
||||
# yield Element()
|
||||
component['body'].append(process_yield(stmt.value))
|
||||
|
||||
return component
|
||||
```
|
||||
|
||||
This gives us:
|
||||
- **Zero parsing overhead** - Python does it for us
|
||||
- **Perfect Python compatibility** - It IS Python
|
||||
- **Error messages for free** - Python's parser handles syntax errors
|
||||
- **Type hints support** - Can extract from AST
|
||||
|
||||
### 2. LLVM IR Generation
|
||||
|
||||
```cpp
|
||||
// codegen.cpp
|
||||
class RitzCodeGen {
|
||||
llvm::LLVMContext context;
|
||||
llvm::IRBuilder<> builder;
|
||||
llvm::Module* module;
|
||||
|
||||
// Compile @component function to WASM
|
||||
llvm::Function* compileComponent(ASTNode* node) {
|
||||
auto func = llvm::Function::Create(
|
||||
llvm::FunctionType::get(builder.getVoidTy(), false),
|
||||
llvm::Function::ExternalLinkage,
|
||||
node->component.name,
|
||||
module
|
||||
);
|
||||
|
||||
auto entry = llvm::BasicBlock::Create(context, "entry", func);
|
||||
builder.SetInsertPoint(entry);
|
||||
|
||||
// Process yields - each becomes DOM operations
|
||||
for (auto stmt : node->component.body) {
|
||||
if (stmt->type == AST_YIELD) {
|
||||
compileYield(stmt);
|
||||
}
|
||||
}
|
||||
|
||||
builder.CreateRetVoid();
|
||||
return func;
|
||||
}
|
||||
|
||||
void compileYield(ASTNode* node) {
|
||||
// yield P('text') compiles to:
|
||||
// 1. Create P element
|
||||
// 2. Create text node
|
||||
// 3. Append text to P
|
||||
// 4. Append P to current parent
|
||||
|
||||
auto create_p = builder.CreateCall(
|
||||
module->getFunction("dom_create_element"),
|
||||
builder.getInt32(TAG_P)
|
||||
);
|
||||
|
||||
// ... rest of DOM operations
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Reactive Runtime (in WASM)
|
||||
|
||||
```c
|
||||
// runtime.c - Compiles to WASM
|
||||
typedef struct {
|
||||
int32_t value;
|
||||
uint32_t subscribers[MAX_SUBSCRIBERS];
|
||||
uint32_t subscriber_count;
|
||||
uint32_t dom_node_id; // Direct DOM node reference
|
||||
} Signal;
|
||||
|
||||
Signal* create_signal(int32_t initial) {
|
||||
Signal* sig = (Signal*)wasm_alloc(sizeof(Signal));
|
||||
sig->value = initial;
|
||||
sig->subscriber_count = 0;
|
||||
return sig;
|
||||
}
|
||||
|
||||
void signal_set(Signal* sig, int32_t value) {
|
||||
sig->value = value;
|
||||
|
||||
// Direct DOM update - no virtual DOM!
|
||||
if (sig->dom_node_id) {
|
||||
dom_ops[op_count++] = OP_UPDATE_TEXT;
|
||||
dom_ops[op_count++] = sig->dom_node_id;
|
||||
dom_ops[op_count++] = (uint32_t)int_to_string(value);
|
||||
}
|
||||
|
||||
// Notify subscribers
|
||||
for (int i = 0; i < sig->subscriber_count; i++) {
|
||||
call_effect(sig->subscribers[i]);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## DOM Bridge Architecture
|
||||
|
||||
### WASM Side - Accumulate Operations
|
||||
|
||||
```c
|
||||
// Opcodes for DOM operations
|
||||
enum DOMOp {
|
||||
OP_CREATE_ELEMENT = 0x01,
|
||||
OP_CREATE_TEXT = 0x02,
|
||||
OP_SET_ATTRIBUTE = 0x03,
|
||||
OP_APPEND_CHILD = 0x04,
|
||||
OP_UPDATE_TEXT = 0x05,
|
||||
OP_ADD_EVENT = 0x06,
|
||||
OP_REMOVE = 0x07,
|
||||
OP_BATCH_END = 0xFF
|
||||
};
|
||||
|
||||
// Global operation buffer
|
||||
uint32_t dom_ops[MAX_OPS];
|
||||
uint32_t op_count = 0;
|
||||
|
||||
// Batch operations then flush
|
||||
void flush_dom_ops() {
|
||||
// Call JS with pointer to operation buffer
|
||||
js_flush_dom_ops(dom_ops, op_count);
|
||||
op_count = 0;
|
||||
}
|
||||
```
|
||||
|
||||
### JavaScript Side - Execute Operations
|
||||
|
||||
```javascript
|
||||
// bridge.js
|
||||
const nodeMap = new Map();
|
||||
let nodeIdCounter = 1;
|
||||
|
||||
function flushDOMOps(ptr, len) {
|
||||
const ops = new Uint32Array(wasmMemory.buffer, ptr, len);
|
||||
const stack = [];
|
||||
|
||||
for (let i = 0; i < len;) {
|
||||
switch(ops[i++]) {
|
||||
case 0x01: { // CREATE_ELEMENT
|
||||
const tagId = ops[i++];
|
||||
const el = document.createElement(TAG_NAMES[tagId]);
|
||||
const id = nodeIdCounter++;
|
||||
nodeMap.set(id, el);
|
||||
stack.push(id);
|
||||
break;
|
||||
}
|
||||
|
||||
case 0x05: { // UPDATE_TEXT
|
||||
const nodeId = ops[i++];
|
||||
const textPtr = ops[i++];
|
||||
const text = readString(textPtr);
|
||||
nodeMap.get(nodeId).textContent = text;
|
||||
break;
|
||||
}
|
||||
|
||||
// ... other operations
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 1: Python Prototype (Week 1)
|
||||
- [x] Define component syntax with yield
|
||||
- [ ] Build Python runtime for testing (framework.py)
|
||||
- [ ] Create HTML generation from component tree
|
||||
- [ ] Signal/reactive state in pure Python
|
||||
|
||||
### Phase 2: Parser (Week 2)
|
||||
- [ ] Python subset grammar definition
|
||||
- [ ] Parser using ANTLR or hand-written recursive descent
|
||||
- [ ] AST generation from Python code
|
||||
- [ ] Semantic analysis (type checking, reactive dependency tracking)
|
||||
|
||||
### Phase 3: WASM Compiler (Week 3-4)
|
||||
- [ ] LLVM setup and IR generation
|
||||
- [ ] Compile components to WASM functions
|
||||
- [ ] Reactive runtime in C/Rust
|
||||
- [ ] DOM operation batching
|
||||
|
||||
### Phase 4: JavaScript Bridge (Week 5)
|
||||
- [ ] DOM operation interpreter in JS
|
||||
- [ ] Event handler callbacks
|
||||
- [ ] String passing optimization
|
||||
- [ ] Memory management
|
||||
|
||||
### Phase 5: Optimization (Week 6)
|
||||
- [ ] Template compilation (static vs dynamic parts)
|
||||
- [ ] Signal-to-DOM direct mapping
|
||||
- [ ] Batch operation compression
|
||||
- [ ] Bundle size optimization
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
### Why Yield?
|
||||
- Natural Python syntax for building trees
|
||||
- Composable via `yield from`
|
||||
- Clear execution order
|
||||
- No magic - just generators
|
||||
|
||||
### Why Compile vs Interpret?
|
||||
- 10-100x performance improvement
|
||||
- 5-10x smaller bundle (no Python runtime)
|
||||
- Direct DOM updates (no virtual DOM)
|
||||
- Type safety at compile time
|
||||
|
||||
### Why LLVM?
|
||||
- Mature toolchain
|
||||
- Excellent WASM backend
|
||||
- Optimization passes
|
||||
- Debugging support
|
||||
|
||||
## Performance Targets
|
||||
|
||||
Based on research of SolidJS, Leptos, and Dioxus:
|
||||
|
||||
- **Bundle size**: < 200KB compressed
|
||||
- **Startup time**: < 500ms on slow 3G
|
||||
- **Update latency**: Within 10% of vanilla JS
|
||||
- **Memory usage**: 70% less than React
|
||||
- **DOM mutations**: 99% fewer than virtual DOM
|
||||
|
||||
## Critical Implementation Details
|
||||
|
||||
### Signal Subscription Model
|
||||
```c
|
||||
// Each signal tracks its subscribers
|
||||
// Each effect tracks its dependencies
|
||||
// Bidirectional relationship for cleanup
|
||||
|
||||
typedef struct Effect {
|
||||
void (*fn)(void);
|
||||
Signal** dependencies;
|
||||
uint32_t dep_count;
|
||||
} Effect;
|
||||
|
||||
// Global context during effect execution
|
||||
Effect* current_effect = NULL;
|
||||
|
||||
// When signal.get() called, register dependency
|
||||
int32_t signal_get(Signal* sig) {
|
||||
if (current_effect) {
|
||||
add_dependency(current_effect, sig);
|
||||
}
|
||||
return sig->value;
|
||||
}
|
||||
```
|
||||
|
||||
### Memory Management
|
||||
- Bump allocator for per-frame allocations
|
||||
- Generation-based GC for long-lived objects
|
||||
- Reference counting for shared resources
|
||||
- No Rc/RefCell exposed to users
|
||||
|
||||
### String Handling
|
||||
- String table in WASM memory
|
||||
- Pass indices, not pointers
|
||||
- Intern common strings
|
||||
- UTF-8 throughout
|
||||
|
||||
## Revolutionary Debugging Story
|
||||
|
||||
### Python Debugging in WASM!
|
||||
|
||||
**THE INSIGHT**: We're not debugging JavaScript - we're debugging Python code running in WASM! This means we can:
|
||||
|
||||
```python
|
||||
# This is what you see in DevTools
|
||||
@component
|
||||
def Counter():
|
||||
count = signal(0) # <-- Breakpoint here shows Python source!
|
||||
|
||||
def increment():
|
||||
count += 1 # <-- Step through Python, not JS!
|
||||
print(f"Count: {count}") # <-- See Python values
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **DWARF Debug Symbols**: LLVM generates standard DWARF symbols in WASM
|
||||
2. **Source Maps on Steroids**: Map WASM instructions → Python source lines
|
||||
3. **Symbol Translation**:
|
||||
```
|
||||
WASM offset 0x1234 → Counter.py:15:8 → "count += 1"
|
||||
```
|
||||
4. **Browser Integration**: Chrome DevTools already supports WASM debugging!
|
||||
5. **Live Editing**: Recompile just the changed component, hot-reload the WASM module
|
||||
|
||||
### Debugging Options
|
||||
|
||||
```javascript
|
||||
// In browser console
|
||||
wasm.debug.setBreakpoint('Counter.py', 15)
|
||||
wasm.debug.stepInto()
|
||||
wasm.debug.inspect('count') // Shows Signal { value: 5, subscribers: [...] }
|
||||
|
||||
// Or even connect Python debugger!
|
||||
# pdb remote debugging over WebSocket
|
||||
import pdb; pdb.set_trace() # This could actually work!
|
||||
```
|
||||
|
||||
## Modular Loading Architecture
|
||||
|
||||
### Progressive WASM Bundles
|
||||
|
||||
Instead of one monolithic WASM file, split into modules:
|
||||
|
||||
```
|
||||
bootstrap.wasm (50KB)
|
||||
├── Minimal runtime
|
||||
├── Login component
|
||||
└── Module loader
|
||||
|
||||
app.wasm (200KB) - Loaded after auth
|
||||
├── Main application
|
||||
├── Core components
|
||||
└── Router
|
||||
|
||||
lazy/
|
||||
├── admin.wasm (150KB) - Admin panel
|
||||
├── analytics.wasm (100KB) - Analytics dashboard
|
||||
├── reports.wasm (120KB) - Report generation
|
||||
└── [feature].wasm - Feature-specific modules
|
||||
```
|
||||
|
||||
### Security Through Architecture
|
||||
|
||||
```python
|
||||
# bootstrap.py - Compiles to bootstrap.wasm
|
||||
@component
|
||||
def LoginPage():
|
||||
with Main():
|
||||
yield LoginForm()
|
||||
# No app code here - not even downloaded!
|
||||
|
||||
# After successful auth
|
||||
async def load_app():
|
||||
# Server only sends this to authenticated users
|
||||
app_module = await fetch('/wasm/app.wasm', {
|
||||
'headers': {'Authorization': f'Bearer {token}'}
|
||||
})
|
||||
wasm.load_module(app_module)
|
||||
```
|
||||
|
||||
### Implementation
|
||||
|
||||
```javascript
|
||||
// module-loader.js
|
||||
class RitzModuleLoader {
|
||||
constructor() {
|
||||
this.modules = new Map();
|
||||
this.cache = new Cache('ritz-modules-v1');
|
||||
}
|
||||
|
||||
async loadModule(name, url) {
|
||||
// Check cache first
|
||||
const cached = await this.cache.match(url);
|
||||
if (cached) {
|
||||
const wasm = await WebAssembly.instantiate(cached);
|
||||
this.modules.set(name, wasm.instance);
|
||||
return;
|
||||
}
|
||||
|
||||
// Download and cache
|
||||
const response = await fetch(url);
|
||||
const bytes = await response.arrayBuffer();
|
||||
|
||||
// Cache for next time
|
||||
await this.cache.put(url, new Response(bytes));
|
||||
|
||||
// Instantiate
|
||||
const wasm = await WebAssembly.instantiate(bytes, {
|
||||
env: this.getImports(name)
|
||||
});
|
||||
|
||||
this.modules.set(name, wasm.instance);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Route-Based Code Splitting
|
||||
|
||||
```python
|
||||
# router.py
|
||||
@component
|
||||
def Router():
|
||||
current_route = signal(window.location.pathname)
|
||||
|
||||
with Main():
|
||||
if current_route == '/admin':
|
||||
# Lazy load admin module
|
||||
yield LazyLoad('admin.wasm', AdminPanel)
|
||||
elif current_route == '/reports':
|
||||
# Only load if needed
|
||||
yield LazyLoad('reports.wasm', ReportsView)
|
||||
else:
|
||||
# Main app is already loaded
|
||||
yield Dashboard()
|
||||
```
|
||||
|
||||
## Next Steps (Updated)
|
||||
|
||||
1. ✅ **Prototype**: Build framework.py with working signals and components
|
||||
2. **AST Parser**: Use Python's `ast` module to parse components
|
||||
3. **Compiler Pipeline**:
|
||||
- Python AST → Ritz IR
|
||||
- Ritz IR → LLVM IR
|
||||
- LLVM IR → WASM with debug symbols
|
||||
4. **Debugging Infrastructure**:
|
||||
- Generate source maps
|
||||
- Embed Python source in WASM
|
||||
- Browser DevTools integration
|
||||
5. **Module System**:
|
||||
- Build tool for splitting modules
|
||||
- Runtime module loader
|
||||
- Caching strategy
|
||||
6. **Test**: Port TodoMVC with lazy-loaded features
|
||||
|
||||
## References
|
||||
|
||||
- SolidJS: Fine-grained reactivity without virtual DOM
|
||||
- Leptos: Rust signals with Copy + 'static
|
||||
- Dioxus Sledgehammer: Batched DOM operations
|
||||
- LLVM WebAssembly Backend: https://llvm.org/docs/WebAssemblyUsage.html
|
||||
- Binaryen: WASM optimizer and code generator
|
||||
|
||||
## Core Innovation
|
||||
|
||||
**Python syntax → WASM performance**. Write and debug in Python, ship as WASM. No virtual DOM, direct signal-to-DOM updates, 99% fewer mutations than React. The future of web frameworks: compiled, reactive, Python-like.
|
||||
Reference in new issue
Block a user