Core API Reference¶
The pydapter.core
module provides the foundational adapter system for converting
between Pydantic models and various data formats. It implements a registry-based
pattern that enables stateless, bidirectional data transformations.
Installation¶
Overview¶
The core module establishes the fundamental concepts of pydapter:
- Adapter Protocol: Defines the interface for data conversion
- Registry System: Manages and discovers adapters
- Adaptable Mixin: Provides convenient model integration
- Error Handling: Comprehensive exception hierarchy for debugging
Core Architecture:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Adapter │ │ AdapterRegistry │ │ Adaptable │
│ (Protocol) │ │ (Manager) │ │ (Mixin) │
│ │ │ │ │ │
│ from_obj() │ │ register() │ │ adapt_from() │
│ to_obj() │ │ get() │ │ adapt_to() │
│ obj_key │ │ adapt_from() │ │ │
└─────────────────┘ │ adapt_to() │ └─────────────────┘
└─────────────────┘
The system supports both synchronous and asynchronous operations through parallel
implementations in pydapter.core
and
pydapter.async_core
.
Core Protocols¶
Adapter¶
Module: pydapter.core
Defines the interface for stateless data conversion between Pydantic models and external formats.
Protocol Interface:
@runtime_checkable
class Adapter(Protocol[T]):
"""Stateless conversion helper."""
obj_key: ClassVar[str] # Unique identifier for the adapter
@classmethod
def from_obj(cls, subj_cls: type[T], obj: Any, /, *, many: bool = False, **kw): ...
@classmethod
def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw): ...
Key Concepts:
- Stateless: Adapters should not maintain internal state
- Bidirectional: Support both
from_obj
(import) andto_obj
(export) - Type-safe: Use generic typing for type safety
- Batch Support: Handle single items or collections via
many
parameter
Implementation Example:
from pydapter.core import Adapter
from pydantic import BaseModel
import json
class JSONAdapter(Adapter):
obj_key = "json"
@classmethod
def from_obj(cls, subj_cls: type[T], obj: str, /, *, many: bool = False, **kw):
"""Convert JSON string to Pydantic model(s)."""
data = json.loads(obj)
if many:
return [subj_cls.model_validate(item) for item in data]
return subj_cls.model_validate(data)
@classmethod
def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw):
"""Convert Pydantic model(s) to JSON string."""
if many or isinstance(subj, list):
data = [item.model_dump() for item in subj]
else:
data = subj.model_dump()
return json.dumps(data, **kw)
# Usage
class User(BaseModel):
name: str
email: str
json_data = '{"name": "John", "email": "john@example.com"}'
user = JSONAdapter.from_obj(User, json_data)
back_to_json = JSONAdapter.to_obj(user)
AsyncAdapter¶
Module: pydapter.async_core
Asynchronous counterpart to the Adapter
protocol for operations requiring async/await.
Protocol Interface:
@runtime_checkable
class AsyncAdapter(Protocol[T]):
"""Stateless, **async** conversion helper."""
obj_key: ClassVar[str]
@classmethod
async def from_obj(
cls, subj_cls: type[T], obj: Any, /, *, many: bool = False, **kw
) -> T | list[T]: ...
@classmethod
async def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw) -> Any: ...
Implementation Example:
from pydapter.async_core import AsyncAdapter
import aiohttp
import json
class HTTPAPIAdapter(AsyncAdapter):
obj_key = "http_api"
@classmethod
async def from_obj(cls, subj_cls: type[T], obj: str, /, *, many: bool = False, **kw):
"""Fetch data from HTTP API and convert to model(s)."""
async with aiohttp.ClientSession() as session:
async with session.get(obj) as response:
data = await response.json()
if many:
return [subj_cls.model_validate(item) for item in data]
return subj_cls.model_validate(data)
@classmethod
async def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw):
"""Convert model(s) to API payload."""
if many or isinstance(subj, list):
return [item.model_dump() for item in subj]
return subj.model_dump()
# Usage
users = await HTTPAPIAdapter.from_obj(User, "https://api.example.com/users", many=True)
Registry System¶
AdapterRegistry¶
Module: pydapter.core
Manages adapter registration and provides convenient access methods.
Class Interface:
class AdapterRegistry:
def __init__(self) -> None: ...
def register(self, adapter_cls: type[Adapter]) -> None: ...
def get(self, obj_key: str) -> type[Adapter]: ...
def adapt_from(self, subj_cls: type[T], obj, *, obj_key: str, **kw): ...
def adapt_to(self, subj, *, obj_key: str, **kw): ...
Usage:
from pydapter.core import AdapterRegistry
# Create registry
registry = AdapterRegistry()
# Register adapters
registry.register(JSONAdapter)
registry.register(CSVAdapter)
# Use via registry
user = registry.adapt_from(User, json_data, obj_key="json")
csv_data = registry.adapt_to(user, obj_key="csv")
# Direct adapter access
adapter_cls = registry.get("json")
user = adapter_cls.from_obj(User, json_data)
Error Handling:
The registry provides comprehensive error handling:
from pydapter.exceptions import AdapterNotFoundError, AdapterError
try:
user = registry.adapt_from(User, data, obj_key="unknown")
except AdapterNotFoundError as e:
print(f"No adapter found: {e}")
except AdapterError as e:
print(f"Adaptation failed: {e}")
AsyncAdapterRegistry¶
Module: pydapter.async_core
Asynchronous version of AdapterRegistry
for async adapters.
Usage:
from pydapter.async_core import AsyncAdapterRegistry
# Create async registry
async_registry = AsyncAdapterRegistry()
async_registry.register(HTTPAPIAdapter)
# Use with async/await
users = await async_registry.adapt_from(User, api_url, obj_key="http_api", many=True)
Adaptable Mixin¶
Adaptable¶
Module: pydapter.core
Mixin class that integrates adapter functionality directly into Pydantic models.
Class Interface:
class Adaptable:
"""Mixin that endows any Pydantic model with adapt-from / adapt-to."""
_adapter_registry: ClassVar[AdapterRegistry | None] = None
@classmethod
def _registry(cls) -> AdapterRegistry: ...
@classmethod
def register_adapter(cls, adapter_cls: type[Adapter]) -> None: ...
@classmethod
def adapt_from(cls, obj, *, obj_key: str, **kw): ...
def adapt_to(self, *, obj_key: str, **kw): ...
Usage:
from pydapter.core import Adaptable
from pydantic import BaseModel
class User(BaseModel, Adaptable):
name: str
email: str
age: int
# Register adapters for this model
User.register_adapter(JSONAdapter)
User.register_adapter(CSVAdapter)
# Use adapter methods directly on the model
user = User.adapt_from(json_data, obj_key="json")
csv_output = user.adapt_to(obj_key="csv")
# Class method for creating from external data
users = User.adapt_from(csv_file_content, obj_key="csv", many=True)
Advanced Usage:
# Custom model with multiple adapters
class Product(BaseModel, Adaptable):
id: str
name: str
price: float
category: str
# Register multiple adapters
Product.register_adapter(JSONAdapter)
Product.register_adapter(XMLAdapter)
Product.register_adapter(DatabaseAdapter)
# Chain conversions
product = Product.adapt_from(xml_data, obj_key="xml")
json_data = product.adapt_to(obj_key="json")
database_record = product.adapt_to(obj_key="database")
Exception Hierarchy¶
Core Exceptions¶
Module: pydapter.exceptions
Comprehensive exception system for error handling and debugging.
AdapterError¶
Base exception for all pydapter errors.
class AdapterError(Exception):
"""Base exception for all pydapter errors."""
def __init__(self, message: str, **context: Any):
super().__init__(message)
self.message = message
self.context = context # Additional error context
Usage:
from pydapter.exceptions import AdapterError
try:
result = adapter.from_obj(Model, invalid_data)
except AdapterError as e:
print(f"Error: {e.message}")
print(f"Context: {e.context}")
ValidationError¶
Exception for data validation failures.
class ValidationError(AdapterError):
"""Exception raised when data validation fails."""
def __init__(self, message: str, data: Optional[Any] = None, **context: Any):
super().__init__(message, **context)
self.data = data # The data that failed validation
TypeConversionError¶
Exception for type conversion failures.
class TypeConversionError(ValidationError):
"""Exception raised when type conversion fails."""
def __init__(
self,
message: str,
source_type: Optional[type] = None,
target_type: Optional[type] = None,
field_name: Optional[str] = None,
model_name: Optional[str] = None,
**context: Any,
): ...
AdapterNotFoundError¶
Exception when no adapter is registered for a given key.
from pydapter.exceptions import AdapterNotFoundError
try:
adapter = registry.get("nonexistent")
except AdapterNotFoundError as e:
print(f"Adapter not found: {e}")
ConfigurationError¶
Exception for adapter configuration issues.
from pydapter.exceptions import ConfigurationError
class BadAdapter:
# Missing obj_key will raise ConfigurationError
pass
try:
registry.register(BadAdapter)
except ConfigurationError as e:
print(f"Configuration error: {e}")
Advanced Usage Patterns¶
Custom Adapter Development¶
Create specialized adapters for specific use cases:
from pydapter.core import Adapter
from typing import Any, TypeVar
import yaml
T = TypeVar("T")
class YAMLAdapter(Adapter[T]):
obj_key = "yaml"
@classmethod
def from_obj(cls, subj_cls: type[T], obj: str, /, *, many: bool = False, **kw):
"""Convert YAML string to Pydantic model(s)."""
data = yaml.safe_load(obj)
if many:
return [subj_cls.model_validate(item) for item in data]
return subj_cls.model_validate(data)
@classmethod
def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw):
"""Convert Pydantic model(s) to YAML string."""
if many or isinstance(subj, list):
data = [item.model_dump() for item in subj]
else:
data = subj.model_dump()
return yaml.dump(data, **kw)
Adapter Composition¶
Combine multiple adapters for complex workflows:
class DataPipeline:
def __init__(self, model_cls, registry: AdapterRegistry):
self.model_cls = model_cls
self.registry = registry
def transform(self, data, from_format: str, to_format: str, **kw):
"""Transform data from one format to another via Pydantic model."""
# Parse input format to model
model_instance = self.registry.adapt_from(
self.model_cls, data, obj_key=from_format, **kw
)
# Convert model to output format
return self.registry.adapt_to(
model_instance, obj_key=to_format, **kw
)
# Usage
pipeline = DataPipeline(User, registry)
json_data = pipeline.transform(csv_data, "csv", "json")
Error Recovery¶
Implement robust error handling with fallbacks:
def safe_adapt_from(model_cls, data, primary_key: str, fallback_key: str, registry):
"""Attempt adaptation with fallback on failure."""
try:
return registry.adapt_from(model_cls, data, obj_key=primary_key)
except AdapterError as e:
print(f"Primary adapter {primary_key} failed: {e}")
try:
return registry.adapt_from(model_cls, data, obj_key=fallback_key)
except AdapterError as fallback_error:
print(f"Fallback adapter {fallback_key} also failed: {fallback_error}")
raise AdapterError(
f"Both {primary_key} and {fallback_key} adapters failed",
primary_error=str(e),
fallback_error=str(fallback_error)
)
# Usage
user = safe_adapt_from(User, data, "json", "yaml", registry)
Best Practices¶
Adapter Design¶
- Stateless Design: Keep adapters stateless for thread safety
- Clear obj_key: Use descriptive, unique keys for adapter identification
- Error Handling: Provide meaningful error messages with context
- Type Safety: Use proper type hints and validation
- Documentation: Document expected input/output formats
Performance Optimization¶
- Lazy Loading: Register adapters only when needed
- Batch Processing: Use
many=True
for collections - Caching: Cache registry lookups for frequently used adapters
- Memory Management: Be mindful of memory usage with large datasets
Registry Management¶
- Global Registry: Use a single global registry for consistency
- Namespace Keys: Use namespaced keys to avoid conflicts (e.g., "db.postgres")
- Validation: Validate adapter implementations before registration
- Testing: Test all registered adapters thoroughly
Error Handling¶
- Specific Exceptions: Use specific exception types for different error conditions
- Context Information: Include relevant context in exception details
- Logging: Log adapter errors for debugging
- Recovery Strategies: Implement fallback mechanisms where appropriate
Integration Examples¶
Database Integration¶
from pydapter.core import Adapter
import sqlite3
class SQLiteAdapter(Adapter):
obj_key = "sqlite"
@classmethod
def from_obj(cls, subj_cls: type[T], obj: str, /, *, many: bool = False, **kw):
"""Load from SQLite database."""
conn = sqlite3.connect(kw.get('database', ':memory:'))
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
if many:
cursor.execute(f"SELECT * FROM {kw.get('table', subj_cls.__name__.lower())}")
rows = cursor.fetchall()
return [subj_cls.model_validate(dict(row)) for row in rows]
else:
cursor.execute(
f"SELECT * FROM {kw.get('table', subj_cls.__name__.lower())} WHERE id = ?",
(kw.get('id'),)
)
row = cursor.fetchone()
return subj_cls.model_validate(dict(row)) if row else None
Web API Integration¶
from pydapter.async_core import AsyncAdapter
import httpx
class RESTAPIAdapter(AsyncAdapter):
obj_key = "rest_api"
@classmethod
async def from_obj(cls, subj_cls: type[T], obj: str, /, *, many: bool = False, **kw):
"""Fetch from REST API."""
async with httpx.AsyncClient() as client:
response = await client.get(obj, params=kw.get('params', {}))
response.raise_for_status()
data = response.json()
if many:
return [subj_cls.model_validate(item) for item in data]
return subj_cls.model_validate(data)
@classmethod
async def to_obj(cls, subj: T | list[T], /, *, many: bool = False, **kw):
"""Post to REST API."""
url = kw.get('url')
if not url:
raise ValueError("URL required for REST API adapter")
async with httpx.AsyncClient() as client:
if many or isinstance(subj, list):
data = [item.model_dump() for item in subj]
else:
data = subj.model_dump()
response = await client.post(url, json=data)
response.raise_for_status()
return response.json()
Migration Guide¶
When upgrading from previous versions:
- Adapter Interface: Update custom adapters to use new protocol interface
- Error Handling: Migrate to new exception hierarchy
- Registry Usage: Use
AdapterRegistry
for better organization - Async Support: Consider migrating to async adapters for I/O operations
- Type Safety: Add proper type hints to existing adapters
For detailed migration instructions, see the Migration Guide.
Auto-generated API Reference¶
The following sections contain auto-generated API documentation:
Core Module¶
pydapter.core
¶
pydapter.core - Adapter protocol, registry, Adaptable mix-in.
Classes¶
Adaptable
¶
Mixin class that adds adapter functionality to Pydantic models.
This mixin provides convenient methods for converting to/from various data formats by maintaining a registry of adapters and providing high-level convenience methods.
When mixed into a Pydantic model, it adds: - Class methods for registering adapters - Class methods for creating instances from external formats - Instance methods for converting to external formats
Example
from pydantic import BaseModel
from pydapter.core import Adaptable
from pydapter.adapters.json_ import JsonAdapter
class Person(BaseModel, Adaptable):
name: str
age: int
# Register an adapter
Person.register_adapter(JsonAdapter)
# Create from JSON
json_data = '{"name": "John", "age": 30}'
person = Person.adapt_from(json_data, obj_key="json")
# Convert to JSON
json_output = person.adapt_to(obj_key="json")
Source code in src/pydapter/core.py
214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 |
|
Functions¶
adapt_from(obj, *, obj_key, **kw)
classmethod
¶
Create model instance(s) from external data format.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
obj
|
Any
|
The source data in the specified format |
required |
obj_key
|
str
|
The key identifying which adapter to use |
required |
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
Any
|
Model instance(s) created from the source data |
Source code in src/pydapter/core.py
adapt_to(*, obj_key, **kw)
¶
Convert this model instance to external data format.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
obj_key
|
str
|
The key identifying which adapter to use |
required |
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
Any
|
Data in the specified external format |
Source code in src/pydapter/core.py
Adapter
¶
Bases: Protocol[T]
Protocol defining the interface for data format adapters.
Adapters are stateless conversion helpers that transform data between Pydantic models and various formats (CSV, JSON, TOML, etc.).
Attributes:
Name | Type | Description |
---|---|---|
obj_key |
str
|
Unique identifier for the adapter type (e.g., "csv", "json") |
Example
from pydantic import BaseModel
from pydapter.adapters.json_ import JsonAdapter
class Person(BaseModel):
name: str
age: int
# Convert from JSON to Pydantic model
json_data = '{"name": "John", "age": 30}'
person = JsonAdapter.from_obj(Person, json_data)
# Convert from Pydantic model to JSON
json_output = JsonAdapter.to_obj(person)
Source code in src/pydapter/core.py
Functions¶
from_obj(subj_cls, obj, /, *, many=False, **kw)
classmethod
¶
Convert from external format to Pydantic model instances.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
subj_cls
|
type[T]
|
The Pydantic model class to instantiate |
required |
obj
|
Any
|
The source data in the adapter's format |
required |
many
|
bool
|
If True, expect/return a list of instances |
False
|
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
T | list[T]
|
Single model instance or list of instances based on 'many' parameter |
Source code in src/pydapter/core.py
to_obj(subj, /, *, many=False, **kw)
classmethod
¶
Convert from Pydantic model instances to external format.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
subj
|
T | list[T]
|
Single model instance or list of instances |
required |
many
|
bool
|
If True, handle as list of instances |
False
|
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
Any
|
Data in the adapter's external format |
Source code in src/pydapter/core.py
AdapterRegistry
¶
Registry for managing and accessing data format adapters.
The registry maintains a mapping of adapter keys to adapter classes, providing a centralized way to register and retrieve adapters for different data formats.
Example
Source code in src/pydapter/core.py
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 |
|
Functions¶
__init__()
¶
adapt_from(subj_cls, obj, *, obj_key, **kw)
¶
Convenience method to convert from external format to Pydantic model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
subj_cls
|
type[T]
|
The Pydantic model class to instantiate |
required |
obj
|
Any
|
The source data in the specified format |
required |
obj_key
|
str
|
The key identifying which adapter to use |
required |
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
T | list[T]
|
Model instance(s) created from the source data |
Raises:
Type | Description |
---|---|
AdapterNotFoundError
|
If no adapter is registered for obj_key |
AdapterError
|
If the adaptation process fails |
Source code in src/pydapter/core.py
adapt_to(subj, *, obj_key, **kw)
¶
Convenience method to convert from Pydantic model to external format.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
subj
|
Any
|
The model instance(s) to convert |
required |
obj_key
|
str
|
The key identifying which adapter to use |
required |
**kw
|
Any
|
Additional adapter-specific arguments |
{}
|
Returns:
Type | Description |
---|---|
Any
|
Data in the specified external format |
Raises:
Type | Description |
---|---|
AdapterNotFoundError
|
If no adapter is registered for obj_key |
AdapterError
|
If the adaptation process fails |
Source code in src/pydapter/core.py
get(obj_key)
¶
Retrieve an adapter class by its key.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
obj_key
|
str
|
The key identifier for the adapter |
required |
Returns:
Type | Description |
---|---|
type[Adapter]
|
The adapter class associated with the key |
Raises:
Type | Description |
---|---|
AdapterNotFoundError
|
If no adapter is registered for the given key |
Source code in src/pydapter/core.py
register(adapter_cls)
¶
Register an adapter class with the registry.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
adapter_cls
|
type[Adapter]
|
The adapter class to register. Must have an 'obj_key' attribute. |
required |
Raises:
Type | Description |
---|---|
ConfigurationError
|
If the adapter class doesn't define 'obj_key' |
Source code in src/pydapter/core.py
Async Core Module¶
pydapter.async_core
¶
pydapter.async_core - async counterparts to the sync Adapter stack
Classes¶
AsyncAdaptable
¶
Mixin that endows any Pydantic model with async adapt-from / adapt-to.
Source code in src/pydapter/async_core.py
AsyncAdapter
¶
Bases: Protocol[T]
Stateless, async conversion helper.
Source code in src/pydapter/async_core.py
Exceptions¶
pydapter.exceptions
¶
pydapter.exceptions - Custom exception hierarchy for pydapter.
Classes¶
AdapterError
¶
Bases: Exception
Base exception for all pydapter errors.
Source code in src/pydapter/exceptions.py
AdapterNotFoundError
¶
Bases: AdapterError
Exception raised when an adapter is not found.
Source code in src/pydapter/exceptions.py
ConfigurationError
¶
Bases: AdapterError
Exception raised when adapter configuration is invalid.
Source code in src/pydapter/exceptions.py
ConnectionError
¶
Bases: AdapterError
Exception raised when a connection to a data source fails.
Source code in src/pydapter/exceptions.py
ParseError
¶
Bases: AdapterError
Exception raised when data parsing fails.
Source code in src/pydapter/exceptions.py
QueryError
¶
Bases: AdapterError
Exception raised when a query to a data source fails.
Source code in src/pydapter/exceptions.py
ResourceError
¶
Bases: AdapterError
Exception raised when a resource (file, database, etc.) cannot be accessed.
Source code in src/pydapter/exceptions.py
TypeConversionError
¶
Bases: ValidationError
Exception raised when type conversion fails.
Source code in src/pydapter/exceptions.py
ValidationError
¶
Bases: AdapterError
Exception raised when data validation fails.