py-orc
Getting Started
Getting Started
What is Orchestrator?
Core Concepts
Pipelines
Tasks
AUTO Tags
Tools
Quick Start Example
Key Features in Action
Input-Agnostic Design
Tool Integration
Model Selection
Next Steps
Installation
Requirements
System Requirements
Optional Requirements
Installation Methods
Using pip (Recommended)
Using conda
Using Docker
Development Installation
Model Setup
Ollama Models (Recommended)
HuggingFace Models
OpenAI/Anthropic Models
Tool Dependencies
Web Tools
System Tools
Data Tools
Configuration
Environment Variables
Verifying Installation
Troubleshooting
Common Issues
Getting Help
Next Steps
Quickstart
Your First Pipeline
Step 1: Create the Pipeline Definition
Step 2: Run the Pipeline
Step 3: Check the Results
Building More Complex Pipelines
Research Report Pipeline
Working with Tools
Available Tool Actions
Using AUTO Tags
Pipeline Composition
Error Handling
Debugging Pipelines
Best Practices
Next Steps
Key Concepts
Pipelines
Input-Agnostic Design
Tasks
Task Anatomy
Task Dependencies
Templates and References
Template Syntax
Runtime vs Compile-Time Resolution
AUTO Tags
How AUTO Tags Work
Tools and Actions
Tool Categories
Action Names
Automatic Tool Detection
Models and Intelligence
Model Types
Intelligent Model Selection
State Management
Checkpointing
Recovery
Control Systems
Built-in Control Systems
Custom Control Systems
Pipeline Composition
Pipeline Imports
Modular Design
Error Handling
Error Strategies
Error Types
Performance Concepts
Parallel Execution
Caching
Resource Management
Security Concepts
Sandboxing
Input Validation
Secret Management
Best Practices
Design Principles
Pipeline Organization
Testing Strategy
Next Steps
User Guide
Tutorials
Web Research Automation
What You’ll Build
Prerequisites
Tutorial 1: Basic Web Search
Step 1: Create the Pipeline
Step 2: Run the Pipeline
Step 3: Understanding the Results
Tutorial 2: Multi-Source Research
Step 1: Multi-Source Pipeline
Step 2: Run Multi-Source Research
Tutorial 3: Fact-Checking Pipeline
Step 1: Fact-Checker Pipeline
Step 2: Use the Fact-Checker
Tutorial 4: Automated Report Generator
Step 1: Report Generator Pipeline
Step 2: Generate Professional Reports
Advanced Exercises
Exercise 1: Industry Monitor
Exercise 2: Competitive Intelligence
Exercise 3: Research Aggregator
Solutions
Next Steps
Tips for Production Use
Data Processing Pipelines
What You’ll Build
Prerequisites
Tutorial 1: Basic ETL Pipeline
Step 1: Create the ETL Pipeline
Step 2: Run the ETL Pipeline
Tutorial 2: Multi-Source Data Integration
Step 1: Multi-Source Integration Pipeline
Step 2: Run Multi-Source Integration
Tutorial 3: Data Quality Assessment Pipeline
Step 1: Data Quality Pipeline
Tutorial 4: Real-Time Data Processing
Step 1: Real-Time Processing Pipeline
Advanced Examples
Example 1: Customer Data Platform
Example 2: Financial Data Pipeline
Exercises
Exercise 1: E-commerce Analytics Pipeline
Exercise 2: IoT Data Processing
Exercise 3: Social Media Analytics
Solutions and Next Steps
Best Practices for Production
Tutorial Overview
Beginner Tutorials
Intermediate Tutorials
Advanced Tutorials
What You’ll Learn
Prerequisites
Tutorial Format
Getting Help
Ready to Start?
API Reference
API Reference
Overview
Core Module (
core
)
Compiler Module (
compiler
)
Models Module (
models
)
Tools Module (
tools
)
Orchestrator Module (
orchestrator
)
Quick Reference
Core Classes
orchestrator.Task
orchestrator.Pipeline
orchestrator.Orchestrator
orchestrator.YAMLCompiler
orchestrator.ModelRegistry
Main Functions
orchestrator.init_models
orchestrator.compile
orchestrator.compile_async
Key Exceptions
Usage Examples
Basic Usage
Advanced Usage
Type Annotations
Environment Variables
Configuration
Performance Considerations
Model Loading
Memory Management
Error Handling
Debugging
Extension Points
Custom Control Systems
Custom Tools
Custom Models
Thread Safety
Testing
Troubleshooting
Common Issues
Getting Help
py-orc
Index
Edit on GitHub
Index
_
|
A
|
C
|
D
|
E
|
F
|
G
|
H
|
I
|
L
|
M
|
N
|
O
|
P
|
R
|
S
|
T
|
U
|
V
|
Y
_
__contains__() (orchestrator.Pipeline method)
__eq__() (orchestrator.Pipeline method)
(orchestrator.Task method)
__hash__() (orchestrator.Pipeline method)
(orchestrator.Task method)
__init__() (orchestrator.ModelRegistry method)
,
[1]
(orchestrator.Orchestrator method)
,
[1]
(orchestrator.Pipeline method)
,
[1]
(orchestrator.Task method)
,
[1]
(orchestrator.YAMLCompiler method)
,
[1]
__iter__() (orchestrator.Pipeline method)
__len__() (orchestrator.Pipeline method)
__post_init__() (orchestrator.Pipeline method)
(orchestrator.Task method)
__repr__() (orchestrator.Orchestrator method)
(orchestrator.Pipeline method)
(orchestrator.Task method)
A
action (orchestrator.Task attribute)
add_task() (orchestrator.Pipeline method)
C
can_retry() (orchestrator.Task method)
clear_execution_history() (orchestrator.Orchestrator method)
clear_tasks() (orchestrator.Pipeline method)
compile() (in module orchestrator)
(orchestrator.YAMLCompiler method)
compile_async() (in module orchestrator)
complete() (orchestrator.Task method)
completed_at (orchestrator.Task attribute)
context (orchestrator.Pipeline attribute)
created_at (orchestrator.Pipeline attribute)
(orchestrator.Task attribute)
D
dependencies (orchestrator.Task attribute)
description (orchestrator.Pipeline attribute)
detect_auto_tags() (orchestrator.YAMLCompiler method)
E
error (orchestrator.Task attribute)
execute_pipeline() (orchestrator.Orchestrator method)
execute_yaml() (orchestrator.Orchestrator method)
execute_yaml_file() (orchestrator.Orchestrator method)
execution_time (orchestrator.Task property)
F
fail() (orchestrator.Task method)
from_dict() (orchestrator.Pipeline class method)
(orchestrator.Task class method)
G
get_available_models() (orchestrator.ModelRegistry method)
get_completed_tasks() (orchestrator.Pipeline method)
get_critical_path() (orchestrator.Pipeline method)
get_dependencies() (orchestrator.Pipeline method)
get_dependents() (orchestrator.Pipeline method)
get_execution_history() (orchestrator.Orchestrator method)
get_execution_levels() (orchestrator.Pipeline method)
get_execution_order() (orchestrator.Pipeline method)
get_execution_order_flat() (orchestrator.Pipeline method)
get_execution_status() (orchestrator.Orchestrator method)
get_failed_tasks() (orchestrator.Pipeline method)
get_model() (orchestrator.ModelRegistry method)
get_model_statistics() (orchestrator.ModelRegistry method)
get_performance_metrics() (orchestrator.Orchestrator method)
get_progress() (orchestrator.Pipeline method)
get_ready_task_ids() (orchestrator.Pipeline method)
get_ready_tasks() (orchestrator.Pipeline method)
get_running_tasks() (orchestrator.Pipeline method)
get_status() (orchestrator.Pipeline method)
get_task() (orchestrator.Pipeline method)
get_task_safe() (orchestrator.Pipeline method)
get_task_strict() (orchestrator.Pipeline method)
get_template_variables() (orchestrator.YAMLCompiler method)
H
has_auto_tags() (orchestrator.YAMLCompiler method)
has_task() (orchestrator.Pipeline method)
health_check() (orchestrator.Orchestrator method)
I
id (orchestrator.Pipeline attribute)
(orchestrator.Task attribute)
init_models() (in module orchestrator)
is_complete() (orchestrator.Pipeline method)
is_failed() (orchestrator.Pipeline method)
is_ready() (orchestrator.Task method)
is_terminal (orchestrator.Task property)
is_valid() (orchestrator.Pipeline method)
L
list_models() (orchestrator.ModelRegistry method)
list_providers() (orchestrator.ModelRegistry method)
list_running_pipelines() (orchestrator.Orchestrator method)
M
max_retries (orchestrator.Task attribute)
metadata (orchestrator.Pipeline attribute)
(orchestrator.Task attribute)
ModelRegistry (class in orchestrator)
N
name (orchestrator.Pipeline attribute)
(orchestrator.Task attribute)
O
Orchestrator (class in orchestrator)
P
parameters (orchestrator.Task attribute)
Pipeline (class in orchestrator)
R
recover_pipeline() (orchestrator.Orchestrator method)
register_model() (orchestrator.ModelRegistry method)
remove_task() (orchestrator.Pipeline method)
remove_task_strict() (orchestrator.Pipeline method)
reset() (orchestrator.Pipeline method)
(orchestrator.Task method)
reset_statistics() (orchestrator.ModelRegistry method)
result (orchestrator.Task attribute)
retry_count (orchestrator.Task attribute)
S
select_model() (orchestrator.ModelRegistry method)
shutdown() (orchestrator.Orchestrator method)
skip() (orchestrator.Task method)
start() (orchestrator.Task method)
started_at (orchestrator.Task attribute)
status (orchestrator.Task attribute)
T
Task (class in orchestrator)
task_count (orchestrator.Pipeline property)
tasks (orchestrator.Pipeline attribute)
timeout (orchestrator.Task attribute)
to_dict() (orchestrator.Pipeline method)
(orchestrator.Task method)
U
unregister_model() (orchestrator.ModelRegistry method)
update_model_performance() (orchestrator.ModelRegistry method)
V
validate_yaml() (orchestrator.YAMLCompiler method)
version (orchestrator.Pipeline attribute)
Y
YAMLCompiler (class in orchestrator)