This commit is contained in:
Gilles Soulier
2026-01-14 21:54:55 +01:00
parent c91c0f1fc9
commit d0b73b9319
140 changed files with 5822 additions and 161 deletions

73
pricewatch.egg-info/PKG-INFO Executable file → Normal file
View File

@@ -28,6 +28,8 @@ Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: redis>=5.0.0
Requires-Dist: rq>=1.15.0
Requires-Dist: rq-scheduler>=0.13.0
Requires-Dist: fastapi>=0.110.0
Requires-Dist: uvicorn>=0.27.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
@@ -100,6 +102,13 @@ pricewatch/
│ │ ├── store.py
│ │ ├── selectors.yml
│ │ └── fixtures/
│ ├── db/ # Persistence SQLAlchemy (Phase 2)
│ │ ├── models.py
│ │ ├── connection.py
│ │ └── migrations/
│ ├── tasks/ # Jobs RQ (Phase 3)
│ │ ├── scrape.py
│ │ └── scheduler.py
│ └── cli/
│ └── main.py # CLI Typer
├── tests/ # Tests pytest
@@ -118,6 +127,9 @@ pricewatch run --yaml scrap_url.yaml --out scraped_store.json
# Avec debug
pricewatch run --yaml scrap_url.yaml --out scraped_store.json --debug
# Avec persistence DB
pricewatch run --yaml scrap_url.yaml --out scraped_store.json --save-db
```
### Commandes utilitaires
@@ -139,6 +151,63 @@ pricewatch parse amazon --in scraped/page.html
pricewatch doctor
```
### Commandes base de donnees
```bash
# Initialiser les tables
pricewatch init-db
# Generer une migration
pricewatch migrate "Initial schema"
# Appliquer les migrations
pricewatch upgrade
# Revenir en arriere
pricewatch downgrade -1
```
### Commandes worker
```bash
# Lancer un worker RQ
pricewatch worker
# Enqueue un job immediat
pricewatch enqueue "https://example.com/product"
# Planifier un job recurrent
pricewatch schedule "https://example.com/product" --interval 24
```
## Base de donnees (Phase 2)
```bash
# Lancer PostgreSQL + Redis en local
docker-compose up -d
# Exemple de configuration
cp .env.example .env
```
Guide de migration JSON -> DB: `MIGRATION_GUIDE.md`
## API REST (Phase 3)
L'API est protegee par un token simple.
```bash
export PW_API_TOKEN=change_me
docker compose up -d api
```
Exemples:
```bash
curl -H "Authorization: Bearer $PW_API_TOKEN" http://localhost:8000/products
curl http://localhost:8000/health
```
## Configuration (scrap_url.yaml)
```yaml
@@ -238,8 +307,8 @@ Aucune erreur ne doit crasher silencieusement : toutes sont loggées et tracées
- ✅ Tests pytest
### Phase 2 : Persistence
- [ ] Base de données PostgreSQL
- [ ] Migrations Alembic
- [x] Base de données PostgreSQL
- [x] Migrations Alembic
- [ ] Historique des prix
### Phase 3 : Automation

View File

@@ -18,6 +18,7 @@ pricewatch/app/core/registry.py
pricewatch/app/core/schema.py
pricewatch/app/scraping/__init__.py
pricewatch/app/scraping/http_fetch.py
pricewatch/app/scraping/pipeline.py
pricewatch/app/scraping/pw_fetch.py
pricewatch/app/stores/__init__.py
pricewatch/app/stores/base.py

View File

@@ -16,6 +16,8 @@ python-dotenv>=1.0.0
redis>=5.0.0
rq>=1.15.0
rq-scheduler>=0.13.0
fastapi>=0.110.0
uvicorn>=0.27.0
[dev]
pytest>=8.0.0