refactor(aieng-agents)!: split aieng-agents into optional [extra] dependencies

[fcogidi]

Summary

Refactors aieng-agents so dependencies are split into optional extras (data, weaviate, code-interpreter, gemini-proxy, news, gradio, observability, plus meta-extra all), with lazy import boundaries and raise_missing_optional-style errors where imports fail without the right extra. Updates aieng-agents/README.md (core vs extras, per-example install hints), CLI bootstrapping for data console scripts, CI workflows (PyPI publish dist path; unit tests actually run with integration-heavy suites skipped), and related tooling/docs alignment.

BREAKING: Default pip install aieng-agents no longer pulls every integration stack; consumers must install 'aieng-agents[all]' or specific extras. Barrel imports from aieng.agents.tools and some root re-exports are removed—use subpackages (see README).

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📝 Documentation update
• 🔧 Refactoring (no functional changes)
• ⚡ Performance improvement
• 🧪 Test improvements
• 🔒 Security fix

Changes Made

• aieng-agents pyproject.toml: Core vs [project.optional-dependencies]; self-referential all extra; explicit deps where code imports directly (e.g. httpx, pydantic-settings).
• Import UX: _optional_extras.py helpers; lazy langfuse package __init__; guarded gradio / data scripts and load_dataset deferred imports; client_manager lazy Weaviate load; tools modules use try/raise + submodule imports (empty tools/__init__.py).
• aieng.agents.cli: Console scripts defer heavy data modules; catch ImportError + ModuleNotFoundError for clear install hints.
• aieng-agents/README.md: Core capabilities, extras table, per-example Requires: lines; corrected import examples and dev/test commands.

Testing

• Tests pass locally — targeted pytest with same ignores as CI (integration/network-heavy tests excluded); full suite may need .env / services.
• Type checking passes (uv run mypy <src_dir>)
• Linting passes (uv run ruff check …) on touched Python paths
• Manual testing performed (describe below)

Deployment Notes

• Consumers of aieng-agents: Pin or bump to new minor/major per semver policy; install aieng-agents[all] (or specific extras) for parity with old monolithic install.
• Lockfile: Run uv lock at repo root if CI expects an updated uv.lock after pyproject.toml changes.

Checklist

• Code follows the project's style guidelines
• Self-review of code completed
• Documentation updated (if applicable)
• No sensitive information (API keys, credentials) exposed

[Copilot]

Pull request overview

Refactors aieng-agents so integration-heavy dependencies are installed via optional extras (e.g. data, weaviate, gradio, observability, etc.), with import boundaries adjusted to fail fast with clearer install guidance when an extra isn’t installed.

Changes:

• Split aieng-agents dependencies into [project.optional-dependencies] extras and updated docs/import paths accordingly.
• Added optional-dependency error helpers + lazy/guarded imports across tools, Langfuse, Gradio, datasets utilities, and console scripts.
• Updated bootcamp implementations/tests to import from subpackages instead of removed barrel exports.

Reviewed changes

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
uv.lock	Updates locked dependency graph to reflect new extras split and version bumps.
tests/tool_tests/test_integration.py	Updates imports to use tool submodules instead of removed aieng.agents.tools barrel exports.
pyproject.toml	Bumps agent-bootcamp version and pins dependency on aieng-agents[all] to match new packaging model.
implementations/3_evals/2_synthetic_data/synthesize_data_e2b.py	Updates CodeInterpreter import to submodule path.
implementations/3_evals/2_synthetic_data/synthesize_data.py	Updates Langfuse + news tool imports to submodules.
implementations/3_evals/1_llm_judge/run_eval.py	Moves setup_langfuse_tracer import to aieng.agents.langfuse (lazy boundary).
implementations/2_frameworks/4_mcp/app.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
implementations/2_frameworks/3_code_interpreter/app.py	Moves Gradio message conversion + CodeInterpreter imports to submodules.
implementations/2_frameworks/2_multi_agent/verbose.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
implementations/2_frameworks/2_multi_agent/efficient_multiple_kbs.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
implementations/2_frameworks/2_multi_agent/efficient.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
implementations/2_frameworks/1_react_rag/langfuse_gradio.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
implementations/2_frameworks/1_react_rag/app.py	Moves Gradio message conversion import to aieng.agents.gradio.messages.
aieng-agents/tests/tools/test_get_news_events.py	Updates tool import to aieng.agents.tools.news_events.
aieng-agents/pyproject.toml	Defines core deps vs optional extras + redirects console scripts through lightweight CLI entrypoints.
aieng-agents/aieng/agents/tools/weaviate_kb.py	Lazily imports weaviate in client factory and improves missing-extra error behavior.
aieng-agents/aieng/agents/tools/news_events.py	Guards BeautifulSoup import and raises extra-specific install hints when missing.
aieng-agents/aieng/agents/tools/code_interpreter.py	Guards E2B imports and raises extra-specific install hints when missing.
aieng-agents/aieng/agents/tools/init.py	Removes barrel exports to avoid importing optional stacks from aieng.agents.tools.
aieng-agents/aieng/agents/tools/README.md	Fixes repo-relative path for running the tool module directly.
aieng-agents/aieng/agents/langfuse/shared_client.py	Guards Langfuse import to enforce [observability] extra.
aieng-agents/aieng/agents/langfuse/oai_sdk_setup.py	Wraps observability imports in an optional-deps guard for clearer failure modes.
aieng-agents/aieng/agents/langfuse/init.py	Implements lazy __getattr__ exports for Langfuse utilities to avoid eager imports.
aieng-agents/aieng/agents/gradio/messages.py	Guards Gradio/PIL imports to enforce [gradio] extra.
aieng-agents/aieng/agents/gradio/init.py	Guards Gradio import to enforce [gradio] extra.
aieng-agents/aieng/agents/data/pdf_to_hf_dataset.py	Guards data-stack imports (click/datasets/dotenv/transformers) behind [data].
aieng-agents/aieng/agents/data/load_dataset.py	Moves datasets/pandas to lazy boundaries and uses optional-deps helper for missing extras.
aieng-agents/aieng/agents/data/chunk_hf_dataset.py	Guards data-stack imports behind [data].
aieng-agents/aieng/agents/client_manager.py	Makes Weaviate integrations lazily loaded to keep core install light.
aieng-agents/aieng/agents/cli.py	Adds thin console-script entrypoints that emit clear “install extra” guidance.
aieng-agents/aieng/agents/agent_session.py	Removes runtime Gradio type import to keep session helper usable without [gradio].
aieng-agents/aieng/agents/_optional_extras.py	Introduces shared helper for consistent missing-extra ImportError messages.
aieng-agents/aieng/agents/init.py	Removes root-level re-exports that would eagerly import optional integrations.
aieng-agents/README.md	Documents core vs extras, updates import examples, and adds install hints per example.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.