commit eedb95afb989c2fd2c8526d2a6c89be8eaa9837b · anil.recoil.org/thicket

+178

README.md

···

       1
       1
       +
       # Thicket

     

       2
       2
       +
       

     

       3
       3
       +
       A modern CLI tool for persisting Atom/RSS feeds in Git repositories, designed to enable distributed webblog comment structures.

     

       4
       4
       +
       

     

       5
       5
       +
       ## Features

     

       6
       6
       +
       

     

       7
       7
       +
       - **Feed Auto-Discovery**: Automatically extracts user metadata from Atom/RSS feeds

     

       8
       8
       +
       - **Git Storage**: Stores feed entries in a Git repository with full history

     

       9
       9
       +
       - **Duplicate Management**: Manual curation of duplicate entries across feeds

     

       10
       10
       +
       - **Modern CLI**: Built with Typer and Rich for beautiful terminal output

     

       11
       11
       +
       - **Comprehensive Parsing**: Supports RSS 0.9x, RSS 1.0, RSS 2.0, and Atom feeds

     

       12
       12
       +
       - **Cron-Friendly**: Designed for scheduled execution

     

       13
       13
       +
       

     

       14
       14
       +
       ## Installation

     

       15
       15
       +
       

     

       16
       16
       +
       ```bash

     

       17
       17
       +
       # Install from source

     

       18
       18
       +
       pip install -e .

     

       19
       19
       +
       

     

       20
       20
       +
       # Or install with dev dependencies

     

       21
       21
       +
       pip install -e .[dev]

     

       22
       22
       +
       ```

     

       23
       23
       +
       

     

       24
       24
       +
       ## Quick Start

     

       25
       25
       +
       

     

       26
       26
       +
       1. **Initialize a new thicket repository:**

     

       27
       27
       +
       ```bash

     

       28
       28
       +
       thicket init ./my-feeds

     

       29
       29
       +
       ```

     

       30
       30
       +
       

     

       31
       31
       +
       2. **Add a user with their feed:**

     

       32
       32
       +
       ```bash

     

       33
       33
       +
       thicket add user "alice" --feed "https://alice.example.com/feed.xml"

     

       34
       34
       +
       ```

     

       35
       35
       +
       

     

       36
       36
       +
       3. **Sync feeds to download entries:**

     

       37
       37
       +
       ```bash

     

       38
       38
       +
       thicket sync --all

     

       39
       39
       +
       ```

     

       40
       40
       +
       

     

       41
       41
       +
       4. **List users and feeds:**

     

       42
       42
       +
       ```bash

     

       43
       43
       +
       thicket list users

     

       44
       44
       +
       thicket list feeds

     

       45
       45
       +
       thicket list entries

     

       46
       46
       +
       ```

     

       47
       47
       +
       

     

       48
       48
       +
       ## Commands

     

       49
       49
       +
       

     

       50
       50
       +
       ### Initialize

     

       51
       51
       +
       ```bash

     

       52
       52
       +
       thicket init <git-store-path> [--cache-dir <path>] [--config <config-file>]

     

       53
       53
       +
       ```

     

       54
       54
       +
       

     

       55
       55
       +
       ### Add Users and Feeds

     

       56
       56
       +
       ```bash

     

       57
       57
       +
       # Add user with auto-discovery

     

       58
       58
       +
       thicket add user "username" --feed "https://example.com/feed.xml"

     

       59
       59
       +
       

     

       60
       60
       +
       # Add user with manual metadata

     

       61
       61
       +
       thicket add user "username" \

     

       62
       62
       +
         --feed "https://example.com/feed.xml" \

     

       63
       63
       +
         --email "user@example.com" \

     

       64
       64
       +
         --homepage "https://example.com" \

     

       65
       65
       +
         --display-name "User Name"

     

       66
       66
       +
       

     

       67
       67
       +
       # Add additional feed to existing user

     

       68
       68
       +
       thicket add feed "username" "https://example.com/other-feed.xml"

     

       69
       69
       +
       ```

     

       70
       70
       +
       

     

       71
       71
       +
       ### Sync Feeds

     

       72
       72
       +
       ```bash

     

       73
       73
       +
       # Sync all users

     

       74
       74
       +
       thicket sync --all

     

       75
       75
       +
       

     

       76
       76
       +
       # Sync specific user

     

       77
       77
       +
       thicket sync --user "username"

     

       78
       78
       +
       

     

       79
       79
       +
       # Dry run (preview changes)

     

       80
       80
       +
       thicket sync --all --dry-run

     

       81
       81
       +
       ```

     

       82
       82
       +
       

     

       83
       83
       +
       ### List Information

     

       84
       84
       +
       ```bash

     

       85
       85
       +
       # List all users

     

       86
       86
       +
       thicket list users

     

       87
       87
       +
       

     

       88
       88
       +
       # List all feeds

     

       89
       89
       +
       thicket list feeds

     

       90
       90
       +
       

     

       91
       91
       +
       # List feeds for specific user

     

       92
       92
       +
       thicket list feeds --user "username"

     

       93
       93
       +
       

     

       94
       94
       +
       # List recent entries

     

       95
       95
       +
       thicket list entries --limit 20

     

       96
       96
       +
       

     

       97
       97
       +
       # List entries for specific user

     

       98
       98
       +
       thicket list entries --user "username"

     

       99
       99
       +
       ```

     

       100
       100
       +
       

     

       101
       101
       +
       ### Manage Duplicates

     

       102
       102
       +
       ```bash

     

       103
       103
       +
       # List duplicate mappings

     

       104
       104
       +
       thicket duplicates list

     

       105
       105
       +
       

     

       106
       106
       +
       # Mark entries as duplicates

     

       107
       107
       +
       thicket duplicates add "https://example.com/dup" "https://example.com/canonical"

     

       108
       108
       +
       

     

       109
       109
       +
       # Remove duplicate mapping

     

       110
       110
       +
       thicket duplicates remove "https://example.com/dup"

     

       111
       111
       +
       ```

     

       112
       112
       +
       

     

       113
       113
       +
       ## Configuration

     

       114
       114
       +
       

     

       115
       115
       +
       Thicket uses a YAML configuration file (default: `thicket.yaml`):

     

       116
       116
       +
       

     

       117
       117
       +
       ```yaml

     

       118
       118
       +
       git_store: ./feeds-repo

     

       119
       119
       +
       cache_dir: ~/.cache/thicket

     

       120
       120
       +
       users:

     

       121
       121
       +
         - username: alice

     

       122
       122
       +
           feeds:

     

       123
       123
       +
             - https://alice.example.com/feed.xml

     

       124
       124
       +
           email: alice@example.com

     

       125
       125
       +
           homepage: https://alice.example.com

     

       126
       126
       +
           display_name: Alice

     

       127
       127
       +
       ```

     

       128
       128
       +
       

     

       129
       129
       +
       ## Git Repository Structure

     

       130
       130
       +
       

     

       131
       131
       +
       ```

     

       132
       132
       +
       feeds-repo/

     

       133
       133
       +
       ├── index.json              # User directory index

     

       134
       134
       +
       ├── duplicates.json         # Duplicate entry mappings

     

       135
       135
       +
       ├── alice/

     

       136
       136
       +
       │   ├── metadata.json       # User metadata

     

       137
       137
       +
       │   ├── entry_id_1.json     # Feed entries

     

       138
       138
       +
       │   └── entry_id_2.json

     

       139
       139
       +
       └── bob/

     

       140
       140
       +
           └── ...

     

       141
       141
       +
       ```

     

       142
       142
       +
       

     

       143
       143
       +
       ## Development

     

       144
       144
       +
       

     

       145
       145
       +
       ### Setup

     

       146
       146
       +
       ```bash

     

       147
       147
       +
       # Install in development mode

     

       148
       148
       +
       pip install -e .[dev]

     

       149
       149
       +
       

     

       150
       150
       +
       # Run tests

     

       151
       151
       +
       pytest

     

       152
       152
       +
       

     

       153
       153
       +
       # Run linting

     

       154
       154
       +
       ruff check src/

     

       155
       155
       +
       black --check src/

     

       156
       156
       +
       

     

       157
       157
       +
       # Run type checking

     

       158
       158
       +
       mypy src/

     

       159
       159
       +
       ```

     

       160
       160
       +
       

     

       161
       161
       +
       ### Architecture

     

       162
       162
       +
       

     

       163
       163
       +
       - **CLI**: Modern interface with Typer and Rich

     

       164
       164
       +
       - **Feed Processing**: Universal parsing with feedparser

     

       165
       165
       +
       - **Git Storage**: Structured storage with GitPython

     

       166
       166
       +
       - **Data Models**: Pydantic for validation and serialization

     

       167
       167
       +
       - **Async HTTP**: httpx for efficient feed fetching

     

       168
       168
       +
       

     

       169
       169
       +
       ## Use Cases

     

       170
       170
       +
       

     

       171
       171
       +
       - **Blog Aggregation**: Collect and archive blog posts from multiple sources

     

       172
       172
       +
       - **Comment Networks**: Enable distributed commenting systems

     

       173
       173
       +
       - **Feed Archival**: Preserve feed history beyond typical feed depth limits

     

       174
       174
       +
       - **Content Curation**: Manage and deduplicate content across feeds

     

       175
       175
       +
       

     

       176
       176
       +
       ## License

     

       177
       177
       +
       

     

       178
       178
       +
       MIT License - see LICENSE file for details.

+169

pyproject.toml

···

       1
       1
       +
       [build-system]

     

       2
       2
       +
       requires = ["hatchling"]

     

       3
       3
       +
       build-backend = "hatchling.build"

     

       4
       4
       +
       

     

       5
       5
       +
       [project]

     

       6
       6
       +
       name = "thicket"

     

       7
       7
       +
       dynamic = ["version"]

     

       8
       8
       +
       description = "A CLI tool for persisting Atom/RSS feeds in Git repositories"

     

       9
       9
       +
       readme = "README.md"

     

       10
       10
       +
       license = "MIT"

     

       11
       11
       +
       requires-python = ">=3.9"

     

       12
       12
       +
       authors = [

     

       13
       13
       +
           {name = "thicket", email = "thicket@example.com"},

     

       14
       14
       +
       ]

     

       15
       15
       +
       classifiers = [

     

       16
       16
       +
           "Development Status :: 3 - Alpha",

     

       17
       17
       +
           "Intended Audience :: Developers",

     

       18
       18
       +
           "License :: OSI Approved :: MIT License",

     

       19
       19
       +
           "Operating System :: OS Independent",

     

       20
       20
       +
           "Programming Language :: Python :: 3",

     

       21
       21
       +
           "Programming Language :: Python :: 3.9",

     

       22
       22
       +
           "Programming Language :: Python :: 3.10",

     

       23
       23
       +
           "Programming Language :: Python :: 3.11",

     

       24
       24
       +
           "Programming Language :: Python :: 3.12",

     

       25
       25
       +
           "Programming Language :: Python :: 3.13",

     

       26
       26
       +
           "Topic :: Internet :: WWW/HTTP :: Dynamic Content :: News/Diary",

     

       27
       27
       +
           "Topic :: Software Development :: Version Control :: Git",

     

       28
       28
       +
           "Topic :: Text Processing :: Markup :: XML",

     

       29
       29
       +
       ]

     

       30
       30
       +
       dependencies = [

     

       31
       31
       +
           "typer>=0.15.0",

     

       32
       32
       +
           "rich>=13.0.0",

     

       33
       33
       +
           "GitPython>=3.1.40",

     

       34
       34
       +
           "feedparser>=6.0.11",

     

       35
       35
       +
           "pydantic>=2.11.0",

     

       36
       36
       +
           "pydantic-settings>=2.10.0",

     

       37
       37
       +
           "httpx>=0.28.0",

     

       38
       38
       +
           "pendulum>=3.0.0",

     

       39
       39
       +
           "bleach>=6.0.0",

     

       40
       40
       +
           "platformdirs>=4.0.0",

     

       41
       41
       +
           "pyyaml>=6.0.0",

     

       42
       42
       +
       ]

     

       43
       43
       +
       

     

       44
       44
       +
       [project.optional-dependencies]

     

       45
       45
       +
       dev = [

     

       46
       46
       +
           "pytest>=8.0.0",

     

       47
       47
       +
           "pytest-asyncio>=0.24.0",

     

       48
       48
       +
           "pytest-cov>=6.0.0",

     

       49
       49
       +
           "black>=24.0.0",

     

       50
       50
       +
           "ruff>=0.8.0",

     

       51
       51
       +
           "mypy>=1.13.0",

     

       52
       52
       +
           "types-PyYAML>=6.0.0",

     

       53
       53
       +
       ]

     

       54
       54
       +
       

     

       55
       55
       +
       [project.urls]

     

       56
       56
       +
       Homepage = "https://github.com/example/thicket"

     

       57
       57
       +
       Documentation = "https://github.com/example/thicket"

     

       58
       58
       +
       Repository = "https://github.com/example/thicket"

     

       59
       59
       +
       "Bug Tracker" = "https://github.com/example/thicket/issues"

     

       60
       60
       +
       

     

       61
       61
       +
       [project.scripts]

     

       62
       62
       +
       thicket = "thicket.cli.main:app"

     

       63
       63
       +
       

     

       64
       64
       +
       [tool.hatch.version]

     

       65
       65
       +
       path = "src/thicket/__init__.py"

     

       66
       66
       +
       

     

       67
       67
       +
       [tool.hatch.build.targets.wheel]

     

       68
       68
       +
       packages = ["src/thicket"]

     

       69
       69
       +
       

     

       70
       70
       +
       [tool.black]

     

       71
       71
       +
       line-length = 88

     

       72
       72
       +
       target-version = ['py39']

     

       73
       73
       +
       include = '\.pyi?$'

     

       74
       74
       +
       extend-exclude = '''

     

       75
       75
       +
       /(

     

       76
       76
       +
         # directories

     

       77
       77
       +
         \.eggs

     

       78
       78
       +
         | \.git

     

       79
       79
       +
         | \.hg

     

       80
       80
       +
         | \.mypy_cache

     

       81
       81
       +
         | \.tox

     

       82
       82
       +
         | \.venv

     

       83
       83
       +
         | build

     

       84
       84
       +
         | dist

     

       85
       85
       +
       )/

     

       86
       86
       +
       '''

     

       87
       87
       +
       

     

       88
       88
       +
       [tool.ruff]

     

       89
       89
       +
       target-version = "py39"

     

       90
       90
       +
       line-length = 88

     

       91
       91
       +
       select = [

     

       92
       92
       +
           "E",  # pycodestyle errors

     

       93
       93
       +
           "W",  # pycodestyle warnings

     

       94
       94
       +
           "F",  # pyflakes

     

       95
       95
       +
           "I",  # isort

     

       96
       96
       +
           "B",  # flake8-bugbear

     

       97
       97
       +
           "C4", # flake8-comprehensions

     

       98
       98
       +
           "UP", # pyupgrade

     

       99
       99
       +
       ]

     

       100
       100
       +
       ignore = [

     

       101
       101
       +
           "E501",  # line too long, handled by black

     

       102
       102
       +
           "B008",  # do not perform function calls in argument defaults

     

       103
       103
       +
           "C901",  # too complex

     

       104
       104
       +
       ]

     

       105
       105
       +
       

     

       106
       106
       +
       [tool.ruff.per-file-ignores]

     

       107
       107
       +
       "__init__.py" = ["F401"]

     

       108
       108
       +
       

     

       109
       109
       +
       [tool.mypy]

     

       110
       110
       +
       python_version = "3.9"

     

       111
       111
       +
       check_untyped_defs = true

     

       112
       112
       +
       disallow_any_generics = true

     

       113
       113
       +
       disallow_incomplete_defs = true

     

       114
       114
       +
       disallow_untyped_defs = true

     

       115
       115
       +
       no_implicit_optional = true

     

       116
       116
       +
       warn_redundant_casts = true

     

       117
       117
       +
       warn_unused_ignores = true

     

       118
       118
       +
       warn_return_any = true

     

       119
       119
       +
       strict_optional = true

     

       120
       120
       +
       

     

       121
       121
       +
       [[tool.mypy.overrides]]

     

       122
       122
       +
       module = [

     

       123
       123
       +
           "feedparser",

     

       124
       124
       +
           "git",

     

       125
       125
       +
           "bleach",

     

       126
       126
       +
       ]

     

       127
       127
       +
       ignore_missing_imports = true

     

       128
       128
       +
       

     

       129
       129
       +
       [tool.pytest.ini_options]

     

       130
       130
       +
       testpaths = ["tests"]

     

       131
       131
       +
       python_files = ["test_*.py"]

     

       132
       132
       +
       python_classes = ["Test*"]

     

       133
       133
       +
       python_functions = ["test_*"]

     

       134
       134
       +
       addopts = [

     

       135
       135
       +
           "-ra",

     

       136
       136
       +
           "--strict-markers",

     

       137
       137
       +
           "--strict-config",

     

       138
       138
       +
           "--cov=src/thicket",

     

       139
       139
       +
           "--cov-report=term-missing",

     

       140
       140
       +
           "--cov-report=html",

     

       141
       141
       +
           "--cov-report=xml",

     

       142
       142
       +
       ]

     

       143
       143
       +
       filterwarnings = [

     

       144
       144
       +
           "error",

     

       145
       145
       +
           "ignore::UserWarning",

     

       146
       146
       +
           "ignore::DeprecationWarning",

     

       147
       147
       +
       ]

     

       148
       148
       +
       markers = [

     

       149
       149
       +
           "slow: marks tests as slow (deselect with '-m \"not slow\"')",

     

       150
       150
       +
           "integration: marks tests as integration tests",

     

       151
       151
       +
       ]

     

       152
       152
       +
       

     

       153
       153
       +
       [tool.coverage.run]

     

       154
       154
       +
       source = ["src"]

     

       155
       155
       +
       branch = true

     

       156
       156
       +
       

     

       157
       157
       +
       [tool.coverage.report]

     

       158
       158
       +
       exclude_lines = [

     

       159
       159
       +
           "pragma: no cover",

     

       160
       160
       +
           "def __repr__",

     

       161
       161
       +
           "if self.debug:",

     

       162
       162
       +
           "if settings.DEBUG",

     

       163
       163
       +
           "raise AssertionError",

     

       164
       164
       +
           "raise NotImplementedError",

     

       165
       165
       +
           "if 0:",

     

       166
       166
       +
           "if __name__ == .__main__.:",

     

       167
       167
       +
           "class .*\\bProtocol\\):",

     

       168
       168
       +
           "@(abc\\.)?abstractmethod",

     

       169
       169
       +
       ]

src/thicket/__init__.py

···

       1
       1
       +
       """Thicket: A CLI tool for persisting Atom/RSS feeds in Git repositories."""

     

       2
       2
       +
       

     

       3
       3
       +
       __version__ = "0.1.0"

     

       4
       4
       +
       __author__ = "thicket"

     

       5
       5
       +
       __email__ = "thicket@example.com"

src/thicket/__main__.py

···

       1
       1
       +
       """Entry point for running thicket as a module."""

     

       2
       2
       +
       

     

       3
       3
       +
       from .cli.main import app

     

       4
       4
       +
       

     

       5
       5
       +
       if __name__ == "__main__":

     

       6
       6
       +
           app()

src/thicket/cli/__init__.py

···

       1
       1
       +
       """CLI interface for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from .main import app

     

       4
       4
       +
       

     

       5
       5
       +
       __all__ = ["app"]

src/thicket/cli/commands/__init__.py

···

       1
       1
       +
       """CLI commands for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       # Import all commands to register them with the main app

     

       4
       4
       +
       from . import add, duplicates, init, list_cmd, sync

     

       5
       5
       +
       

     

       6
       6
       +
       __all__ = ["add", "duplicates", "init", "list_cmd", "sync"]

+193

src/thicket/cli/commands/add.py

···

       1
       1
       +
       """Add command for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       import asyncio

     

       4
       4
       +
       from pathlib import Path

     

       5
       5
       +
       from typing import Optional

     

       6
       6
       +
       

     

       7
       7
       +
       import typer

     

       8
       8
       +
       from pydantic import HttpUrl, ValidationError

     

       9
       9
       +
       

     

       10
       10
       +
       from ...core.feed_parser import FeedParser

     

       11
       11
       +
       from ...core.git_store import GitStore

     

       12
       12
       +
       from ...models import UserConfig

     

       13
       13
       +
       from ..main import app

     

       14
       14
       +
       from ..utils import (

     

       15
       15
       +
           create_progress,

     

       16
       16
       +
           load_config,

     

       17
       17
       +
           print_error,

     

       18
       18
       +
           print_info,

     

       19
       19
       +
           print_success,

     

       20
       20
       +
           save_config,

     

       21
       21
       +
       )

     

       22
       22
       +
       

     

       23
       23
       +
       

     

       24
       24
       +
       @app.command("add")

     

       25
       25
       +
       def add_command(

     

       26
       26
       +
           subcommand: str = typer.Argument(..., help="Subcommand: 'user' or 'feed'"),

     

       27
       27
       +
           username: str = typer.Argument(..., help="Username"),

     

       28
       28
       +
           feed_url: Optional[str] = typer.Argument(None, help="Feed URL (required for 'user' command)"),

     

       29
       29
       +
           email: Optional[str] = typer.Option(None, "--email", "-e", help="User email"),

     

       30
       30
       +
           homepage: Optional[str] = typer.Option(None, "--homepage", "-h", help="User homepage"),

     

       31
       31
       +
           icon: Optional[str] = typer.Option(None, "--icon", "-i", help="User icon URL"),

     

       32
       32
       +
           display_name: Optional[str] = typer.Option(None, "--display-name", "-d", help="User display name"),

     

       33
       33
       +
           config_file: Optional[Path] = typer.Option(

     

       34
       34
       +
               Path("thicket.yaml"), "--config", help="Configuration file path"

     

       35
       35
       +
           ),

     

       36
       36
       +
           auto_discover: bool = typer.Option(

     

       37
       37
       +
               True, "--auto-discover/--no-auto-discover", help="Auto-discover user metadata from feed"

     

       38
       38
       +
           ),

     

       39
       39
       +
       ) -> None:

     

       40
       40
       +
           """Add a user or feed to thicket."""

     

       41
       41
       +
           

     

       42
       42
       +
           if subcommand == "user":

     

       43
       43
       +
               add_user(username, feed_url, email, homepage, icon, display_name, config_file, auto_discover)

     

       44
       44
       +
           elif subcommand == "feed":

     

       45
       45
       +
               add_feed(username, feed_url, config_file)

     

       46
       46
       +
           else:

     

       47
       47
       +
               print_error(f"Unknown subcommand: {subcommand}")

     

       48
       48
       +
               print_error("Use 'user' or 'feed'")

     

       49
       49
       +
               raise typer.Exit(1)

     

       50
       50
       +
       

     

       51
       51
       +
       

     

       52
       52
       +
       def add_user(

     

       53
       53
       +
           username: str,

     

       54
       54
       +
           feed_url: Optional[str],

     

       55
       55
       +
           email: Optional[str],

     

       56
       56
       +
           homepage: Optional[str],

     

       57
       57
       +
           icon: Optional[str],

     

       58
       58
       +
           display_name: Optional[str],

     

       59
       59
       +
           config_file: Path,

     

       60
       60
       +
           auto_discover: bool,

     

       61
       61
       +
       ) -> None:

     

       62
       62
       +
           """Add a new user with feed."""

     

       63
       63
       +
           

     

       64
       64
       +
           if not feed_url:

     

       65
       65
       +
               print_error("Feed URL is required when adding a user")

     

       66
       66
       +
               raise typer.Exit(1)

     

       67
       67
       +
           

     

       68
       68
       +
           # Validate feed URL

     

       69
       69
       +
           try:

     

       70
       70
       +
               validated_feed_url = HttpUrl(feed_url)

     

       71
       71
       +
           except ValidationError:

     

       72
       72
       +
               print_error(f"Invalid feed URL: {feed_url}")

     

       73
       73
       +
               raise typer.Exit(1)

     

       74
       74
       +
           

     

       75
       75
       +
           # Load configuration

     

       76
       76
       +
           config = load_config(config_file)

     

       77
       77
       +
           

     

       78
       78
       +
           # Check if user already exists

     

       79
       79
       +
           existing_user = config.find_user(username)

     

       80
       80
       +
           if existing_user:

     

       81
       81
       +
               print_error(f"User '{username}' already exists")

     

       82
       82
       +
               print_error("Use 'thicket add feed' to add additional feeds")

     

       83
       83
       +
               raise typer.Exit(1)

     

       84
       84
       +
           

     

       85
       85
       +
           # Auto-discover metadata if enabled

     

       86
       86
       +
           discovered_metadata = None

     

       87
       87
       +
           if auto_discover:

     

       88
       88
       +
               discovered_metadata = asyncio.run(discover_feed_metadata(validated_feed_url))

     

       89
       89
       +
           

     

       90
       90
       +
           # Create user config with manual overrides taking precedence

     

       91
       91
       +
           user_config = UserConfig(

     

       92
       92
       +
               username=username,

     

       93
       93
       +
               feeds=[validated_feed_url],

     

       94
       94
       +
               email=email or (discovered_metadata.author_email if discovered_metadata else None),

     

       95
       95
       +
               homepage=HttpUrl(homepage) if homepage else (discovered_metadata.author_uri or discovered_metadata.link if discovered_metadata else None),

     

       96
       96
       +
               icon=HttpUrl(icon) if icon else (discovered_metadata.logo or discovered_metadata.icon or discovered_metadata.image_url if discovered_metadata else None),

     

       97
       97
       +
               display_name=display_name or (discovered_metadata.author_name or discovered_metadata.title if discovered_metadata else None),

     

       98
       98
       +
           )

     

       99
       99
       +
           

     

       100
       100
       +
           # Add user to configuration

     

       101
       101
       +
           config.add_user(user_config)

     

       102
       102
       +
           

     

       103
       103
       +
           # Save configuration

     

       104
       104
       +
           save_config(config, config_file)

     

       105
       105
       +
           

     

       106
       106
       +
           # Add user to Git store

     

       107
       107
       +
           git_store = GitStore(config.git_store)

     

       108
       108
       +
           git_store.add_user(

     

       109
       109
       +
               username=username,

     

       110
       110
       +
               display_name=user_config.display_name,

     

       111
       111
       +
               email=user_config.email,

     

       112
       112
       +
               homepage=str(user_config.homepage) if user_config.homepage else None,

     

       113
       113
       +
               icon=str(user_config.icon) if user_config.icon else None,

     

       114
       114
       +
               feeds=[str(f) for f in user_config.feeds],

     

       115
       115
       +
           )

     

       116
       116
       +
           

     

       117
       117
       +
           # Commit changes

     

       118
       118
       +
           git_store.commit_changes(f"Add user: {username}")

     

       119
       119
       +
           

     

       120
       120
       +
           print_success(f"Added user '{username}' with feed: {feed_url}")

     

       121
       121
       +
           

     

       122
       122
       +
           if discovered_metadata and auto_discover:

     

       123
       123
       +
               print_info("Auto-discovered metadata:")

     

       124
       124
       +
               if user_config.display_name:

     

       125
       125
       +
                   print_info(f"  Display name: {user_config.display_name}")

     

       126
       126
       +
               if user_config.email:

     

       127
       127
       +
                   print_info(f"  Email: {user_config.email}")

     

       128
       128
       +
               if user_config.homepage:

     

       129
       129
       +
                   print_info(f"  Homepage: {user_config.homepage}")

     

       130
       130
       +
               if user_config.icon:

     

       131
       131
       +
                   print_info(f"  Icon: {user_config.icon}")

     

       132
       132
       +
       

     

       133
       133
       +
       

     

       134
       134
       +
       def add_feed(username: str, feed_url: Optional[str], config_file: Path) -> None:

     

       135
       135
       +
           """Add a feed to an existing user."""

     

       136
       136
       +
           

     

       137
       137
       +
           if not feed_url:

     

       138
       138
       +
               print_error("Feed URL is required")

     

       139
       139
       +
               raise typer.Exit(1)

     

       140
       140
       +
           

     

       141
       141
       +
           # Validate feed URL

     

       142
       142
       +
           try:

     

       143
       143
       +
               validated_feed_url = HttpUrl(feed_url)

     

       144
       144
       +
           except ValidationError:

     

       145
       145
       +
               print_error(f"Invalid feed URL: {feed_url}")

     

       146
       146
       +
               raise typer.Exit(1)

     

       147
       147
       +
           

     

       148
       148
       +
           # Load configuration

     

       149
       149
       +
           config = load_config(config_file)

     

       150
       150
       +
           

     

       151
       151
       +
           # Check if user exists

     

       152
       152
       +
           user = config.find_user(username)

     

       153
       153
       +
           if not user:

     

       154
       154
       +
               print_error(f"User '{username}' not found")

     

       155
       155
       +
               print_error("Use 'thicket add user' to add a new user")

     

       156
       156
       +
               raise typer.Exit(1)

     

       157
       157
       +
           

     

       158
       158
       +
           # Check if feed already exists

     

       159
       159
       +
           if validated_feed_url in user.feeds:

     

       160
       160
       +
               print_error(f"Feed already exists for user '{username}': {feed_url}")

     

       161
       161
       +
               raise typer.Exit(1)

     

       162
       162
       +
           

     

       163
       163
       +
           # Add feed to user

     

       164
       164
       +
           if config.add_feed_to_user(username, validated_feed_url):

     

       165
       165
       +
               save_config(config, config_file)

     

       166
       166
       +
               

     

       167
       167
       +
               # Update Git store

     

       168
       168
       +
               git_store = GitStore(config.git_store)

     

       169
       169
       +
               git_store.update_user(username, feeds=[str(f) for f in user.feeds])

     

       170
       170
       +
               git_store.commit_changes(f"Add feed to user {username}: {feed_url}")

     

       171
       171
       +
               

     

       172
       172
       +
               print_success(f"Added feed to user '{username}': {feed_url}")

     

       173
       173
       +
           else:

     

       174
       174
       +
               print_error(f"Failed to add feed to user '{username}'")

     

       175
       175
       +
               raise typer.Exit(1)

     

       176
       176
       +
       

     

       177
       177
       +
       

     

       178
       178
       +
       async def discover_feed_metadata(feed_url: HttpUrl):

     

       179
       179
       +
           """Discover metadata from a feed URL."""

     

       180
       180
       +
           try:

     

       181
       181
       +
               with create_progress() as progress:

     

       182
       182
       +
                   task = progress.add_task("Discovering feed metadata...", total=None)

     

       183
       183
       +
                   

     

       184
       184
       +
                   parser = FeedParser()

     

       185
       185
       +
                   content = await parser.fetch_feed(feed_url)

     

       186
       186
       +
                   metadata, _ = parser.parse_feed(content, feed_url)

     

       187
       187
       +
                   

     

       188
       188
       +
                   progress.update(task, completed=True)

     

       189
       189
       +
                   return metadata

     

       190
       190
       +
                   

     

       191
       191
       +
           except Exception as e:

     

       192
       192
       +
               print_error(f"Failed to discover feed metadata: {e}")

     

       193
       193
       +
               return None

+121

src/thicket/cli/commands/duplicates.py

···

       1
       1
       +
       """Duplicates command for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from pathlib import Path

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       import typer

     

       7
       7
       +
       from rich.table import Table

     

       8
       8
       +
       

     

       9
       9
       +
       from ...core.git_store import GitStore

     

       10
       10
       +
       from ..main import app

     

       11
       11
       +
       from ..utils import (

     

       12
       12
       +
           console,

     

       13
       13
       +
           load_config,

     

       14
       14
       +
           print_error,

     

       15
       15
       +
           print_info,

     

       16
       16
       +
           print_success,

     

       17
       17
       +
       )

     

       18
       18
       +
       

     

       19
       19
       +
       

     

       20
       20
       +
       @app.command("duplicates")

     

       21
       21
       +
       def duplicates_command(

     

       22
       22
       +
           action: str = typer.Argument(..., help="Action: 'list', 'add', 'remove'"),

     

       23
       23
       +
           duplicate_id: Optional[str] = typer.Argument(None, help="Duplicate entry ID"),

     

       24
       24
       +
           canonical_id: Optional[str] = typer.Argument(None, help="Canonical entry ID"),

     

       25
       25
       +
           config_file: Optional[Path] = typer.Option(

     

       26
       26
       +
               Path("thicket.yaml"), "--config", help="Configuration file path"

     

       27
       27
       +
           ),

     

       28
       28
       +
       ) -> None:

     

       29
       29
       +
           """Manage duplicate entry mappings."""

     

       30
       30
       +
           

     

       31
       31
       +
           # Load configuration

     

       32
       32
       +
           config = load_config(config_file)

     

       33
       33
       +
           

     

       34
       34
       +
           # Initialize Git store

     

       35
       35
       +
           git_store = GitStore(config.git_store)

     

       36
       36
       +
           

     

       37
       37
       +
           if action == "list":

     

       38
       38
       +
               list_duplicates(git_store)

     

       39
       39
       +
           elif action == "add":

     

       40
       40
       +
               add_duplicate(git_store, duplicate_id, canonical_id)

     

       41
       41
       +
           elif action == "remove":

     

       42
       42
       +
               remove_duplicate(git_store, duplicate_id)

     

       43
       43
       +
           else:

     

       44
       44
       +
               print_error(f"Unknown action: {action}")

     

       45
       45
       +
               print_error("Use 'list', 'add', or 'remove'")

     

       46
       46
       +
               raise typer.Exit(1)

     

       47
       47
       +
       

     

       48
       48
       +
       

     

       49
       49
       +
       def list_duplicates(git_store: GitStore) -> None:

     

       50
       50
       +
           """List all duplicate mappings."""

     

       51
       51
       +
           duplicates = git_store.get_duplicates()

     

       52
       52
       +
           

     

       53
       53
       +
           if not duplicates.duplicates:

     

       54
       54
       +
               print_info("No duplicate mappings found")

     

       55
       55
       +
               return

     

       56
       56
       +
           

     

       57
       57
       +
           table = Table(title="Duplicate Entry Mappings")

     

       58
       58
       +
           table.add_column("Duplicate ID", style="red")

     

       59
       59
       +
           table.add_column("Canonical ID", style="green")

     

       60
       60
       +
           

     

       61
       61
       +
           for duplicate_id, canonical_id in duplicates.duplicates.items():

     

       62
       62
       +
               table.add_row(duplicate_id, canonical_id)

     

       63
       63
       +
           

     

       64
       64
       +
           console.print(table)

     

       65
       65
       +
           print_info(f"Total duplicates: {len(duplicates.duplicates)}")

     

       66
       66
       +
       

     

       67
       67
       +
       

     

       68
       68
       +
       def add_duplicate(git_store: GitStore, duplicate_id: Optional[str], canonical_id: Optional[str]) -> None:

     

       69
       69
       +
           """Add a duplicate mapping."""

     

       70
       70
       +
           if not duplicate_id:

     

       71
       71
       +
               print_error("Duplicate ID is required")

     

       72
       72
       +
               raise typer.Exit(1)

     

       73
       73
       +
           

     

       74
       74
       +
           if not canonical_id:

     

       75
       75
       +
               print_error("Canonical ID is required")

     

       76
       76
       +
               raise typer.Exit(1)

     

       77
       77
       +
           

     

       78
       78
       +
           # Check if duplicate_id already exists

     

       79
       79
       +
           duplicates = git_store.get_duplicates()

     

       80
       80
       +
           if duplicates.is_duplicate(duplicate_id):

     

       81
       81
       +
               existing_canonical = duplicates.get_canonical(duplicate_id)

     

       82
       82
       +
               print_error(f"Duplicate ID already mapped to: {existing_canonical}")

     

       83
       83
       +
               print_error("Use 'remove' first to change the mapping")

     

       84
       84
       +
               raise typer.Exit(1)

     

       85
       85
       +
           

     

       86
       86
       +
           # Check if we're trying to make a canonical ID point to itself

     

       87
       87
       +
           if duplicate_id == canonical_id:

     

       88
       88
       +
               print_error("Duplicate ID cannot be the same as canonical ID")

     

       89
       89
       +
               raise typer.Exit(1)

     

       90
       90
       +
           

     

       91
       91
       +
           # Add the mapping

     

       92
       92
       +
           git_store.add_duplicate(duplicate_id, canonical_id)

     

       93
       93
       +
           

     

       94
       94
       +
           # Commit changes

     

       95
       95
       +
           git_store.commit_changes(f"Add duplicate mapping: {duplicate_id} -> {canonical_id}")

     

       96
       96
       +
           

     

       97
       97
       +
           print_success(f"Added duplicate mapping: {duplicate_id} -> {canonical_id}")

     

       98
       98
       +
       

     

       99
       99
       +
       

     

       100
       100
       +
       def remove_duplicate(git_store: GitStore, duplicate_id: Optional[str]) -> None:

     

       101
       101
       +
           """Remove a duplicate mapping."""

     

       102
       102
       +
           if not duplicate_id:

     

       103
       103
       +
               print_error("Duplicate ID is required")

     

       104
       104
       +
               raise typer.Exit(1)

     

       105
       105
       +
           

     

       106
       106
       +
           # Check if mapping exists

     

       107
       107
       +
           duplicates = git_store.get_duplicates()

     

       108
       108
       +
           if not duplicates.is_duplicate(duplicate_id):

     

       109
       109
       +
               print_error(f"No duplicate mapping found for: {duplicate_id}")

     

       110
       110
       +
               raise typer.Exit(1)

     

       111
       111
       +
           

     

       112
       112
       +
           canonical_id = duplicates.get_canonical(duplicate_id)

     

       113
       113
       +
           

     

       114
       114
       +
           # Remove the mapping

     

       115
       115
       +
           if git_store.remove_duplicate(duplicate_id):

     

       116
       116
       +
               # Commit changes

     

       117
       117
       +
               git_store.commit_changes(f"Remove duplicate mapping: {duplicate_id} -> {canonical_id}")

     

       118
       118
       +
               print_success(f"Removed duplicate mapping: {duplicate_id} -> {canonical_id}")

     

       119
       119
       +
           else:

     

       120
       120
       +
               print_error(f"Failed to remove duplicate mapping: {duplicate_id}")

     

       121
       121
       +
               raise typer.Exit(1)

+77

src/thicket/cli/commands/init.py

···

       1
       1
       +
       """Initialize command for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from pathlib import Path

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       import typer

     

       7
       7
       +
       from pydantic import ValidationError

     

       8
       8
       +
       

     

       9
       9
       +
       from ...core.git_store import GitStore

     

       10
       10
       +
       from ...models import ThicketConfig

     

       11
       11
       +
       from ..main import app

     

       12
       12
       +
       from ..utils import print_error, print_success, save_config

     

       13
       13
       +
       

     

       14
       14
       +
       

     

       15
       15
       +
       @app.command()

     

       16
       16
       +
       def init(

     

       17
       17
       +
           git_store: Path = typer.Argument(..., help="Path to Git repository for storing feeds"),

     

       18
       18
       +
           cache_dir: Optional[Path] = typer.Option(

     

       19
       19
       +
               None, "--cache-dir", "-c", help="Cache directory (default: ~/.cache/thicket)"

     

       20
       20
       +
           ),

     

       21
       21
       +
           config_file: Optional[Path] = typer.Option(

     

       22
       22
       +
               None, "--config", help="Configuration file path (default: thicket.yaml)"

     

       23
       23
       +
           ),

     

       24
       24
       +
           force: bool = typer.Option(

     

       25
       25
       +
               False, "--force", "-f", help="Overwrite existing configuration"

     

       26
       26
       +
           ),

     

       27
       27
       +
       ) -> None:

     

       28
       28
       +
           """Initialize a new thicket configuration and Git store."""

     

       29
       29
       +
           

     

       30
       30
       +
           # Set default paths

     

       31
       31
       +
           if cache_dir is None:

     

       32
       32
       +
               from platformdirs import user_cache_dir

     

       33
       33
       +
               cache_dir = Path(user_cache_dir("thicket"))

     

       34
       34
       +
           

     

       35
       35
       +
           if config_file is None:

     

       36
       36
       +
               config_file = Path("thicket.yaml")

     

       37
       37
       +
           

     

       38
       38
       +
           # Check if config already exists

     

       39
       39
       +
           if config_file.exists() and not force:

     

       40
       40
       +
               print_error(f"Configuration file already exists: {config_file}")

     

       41
       41
       +
               print_error("Use --force to overwrite")

     

       42
       42
       +
               raise typer.Exit(1)

     

       43
       43
       +
           

     

       44
       44
       +
           # Create cache directory

     

       45
       45
       +
           cache_dir.mkdir(parents=True, exist_ok=True)

     

       46
       46
       +
           

     

       47
       47
       +
           # Create Git store

     

       48
       48
       +
           try:

     

       49
       49
       +
               git_store_obj = GitStore(git_store)

     

       50
       50
       +
               print_success(f"Initialized Git store at: {git_store}")

     

       51
       51
       +
           except Exception as e:

     

       52
       52
       +
               print_error(f"Failed to initialize Git store: {e}")

     

       53
       53
       +
               raise typer.Exit(1)

     

       54
       54
       +
           

     

       55
       55
       +
           # Create configuration

     

       56
       56
       +
           try:

     

       57
       57
       +
               config = ThicketConfig(

     

       58
       58
       +
                   git_store=git_store,

     

       59
       59
       +
                   cache_dir=cache_dir,

     

       60
       60
       +
                   users=[]

     

       61
       61
       +
               )

     

       62
       62
       +
               

     

       63
       63
       +
               save_config(config, config_file)

     

       64
       64
       +
               print_success(f"Created configuration file: {config_file}")

     

       65
       65
       +
               

     

       66
       66
       +
           except ValidationError as e:

     

       67
       67
       +
               print_error(f"Invalid configuration: {e}")

     

       68
       68
       +
               raise typer.Exit(1)

     

       69
       69
       +
           except Exception as e:

     

       70
       70
       +
               print_error(f"Failed to create configuration: {e}")

     

       71
       71
       +
               raise typer.Exit(1)

     

       72
       72
       +
           

     

       73
       73
       +
           print_success("Thicket initialized successfully!")

     

       74
       74
       +
           print_success(f"Git store: {git_store}")

     

       75
       75
       +
           print_success(f"Cache directory: {cache_dir}")

     

       76
       76
       +
           print_success(f"Configuration: {config_file}")

     

       77
       77
       +
           print_success("Run 'thicket add user' to add your first user and feed.")

+146

src/thicket/cli/commands/list_cmd.py

···

       1
       1
       +
       """List command for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from pathlib import Path

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       import typer

     

       7
       7
       +
       from rich.table import Table

     

       8
       8
       +
       

     

       9
       9
       +
       from ...core.git_store import GitStore

     

       10
       10
       +
       from ..main import app

     

       11
       11
       +
       from ..utils import (

     

       12
       12
       +
           console,

     

       13
       13
       +
           load_config,

     

       14
       14
       +
           print_error,

     

       15
       15
       +
           print_feeds_table,

     

       16
       16
       +
           print_info,

     

       17
       17
       +
           print_users_table,

     

       18
       18
       +
       )

     

       19
       19
       +
       

     

       20
       20
       +
       

     

       21
       21
       +
       @app.command("list")

     

       22
       22
       +
       def list_command(

     

       23
       23
       +
           what: str = typer.Argument(..., help="What to list: 'users', 'feeds', 'entries'"),

     

       24
       24
       +
           user: Optional[str] = typer.Option(

     

       25
       25
       +
               None, "--user", "-u", help="Filter by specific user"

     

       26
       26
       +
           ),

     

       27
       27
       +
           limit: Optional[int] = typer.Option(

     

       28
       28
       +
               None, "--limit", "-l", help="Limit number of results"

     

       29
       29
       +
           ),

     

       30
       30
       +
           config_file: Optional[Path] = typer.Option(

     

       31
       31
       +
               Path("thicket.yaml"), "--config", help="Configuration file path"

     

       32
       32
       +
           ),

     

       33
       33
       +
       ) -> None:

     

       34
       34
       +
           """List users, feeds, or entries."""

     

       35
       35
       +
           

     

       36
       36
       +
           # Load configuration

     

       37
       37
       +
           config = load_config(config_file)

     

       38
       38
       +
           

     

       39
       39
       +
           if what == "users":

     

       40
       40
       +
               list_users(config)

     

       41
       41
       +
           elif what == "feeds":

     

       42
       42
       +
               list_feeds(config, user)

     

       43
       43
       +
           elif what == "entries":

     

       44
       44
       +
               list_entries(config, user, limit)

     

       45
       45
       +
           else:

     

       46
       46
       +
               print_error(f"Unknown list type: {what}")

     

       47
       47
       +
               print_error("Use 'users', 'feeds', or 'entries'")

     

       48
       48
       +
               raise typer.Exit(1)

     

       49
       49
       +
       

     

       50
       50
       +
       

     

       51
       51
       +
       def list_users(config) -> None:

     

       52
       52
       +
           """List all users."""

     

       53
       53
       +
           if not config.users:

     

       54
       54
       +
               print_info("No users configured")

     

       55
       55
       +
               return

     

       56
       56
       +
           

     

       57
       57
       +
           print_users_table(config)

     

       58
       58
       +
       

     

       59
       59
       +
       

     

       60
       60
       +
       def list_feeds(config, username: Optional[str] = None) -> None:

     

       61
       61
       +
           """List feeds, optionally filtered by user."""

     

       62
       62
       +
           if username:

     

       63
       63
       +
               user = config.find_user(username)

     

       64
       64
       +
               if not user:

     

       65
       65
       +
                   print_error(f"User '{username}' not found")

     

       66
       66
       +
                   raise typer.Exit(1)

     

       67
       67
       +
               

     

       68
       68
       +
               if not user.feeds:

     

       69
       69
       +
                   print_info(f"No feeds configured for user '{username}'")

     

       70
       70
       +
                   return

     

       71
       71
       +
           

     

       72
       72
       +
           print_feeds_table(config, username)

     

       73
       73
       +
       

     

       74
       74
       +
       

     

       75
       75
       +
       def list_entries(config, username: Optional[str] = None, limit: Optional[int] = None) -> None:

     

       76
       76
       +
           """List entries, optionally filtered by user."""

     

       77
       77
       +
           

     

       78
       78
       +
           # Initialize Git store

     

       79
       79
       +
           git_store = GitStore(config.git_store)

     

       80
       80
       +
           

     

       81
       81
       +
           if username:

     

       82
       82
       +
               # List entries for specific user

     

       83
       83
       +
               user = config.find_user(username)

     

       84
       84
       +
               if not user:

     

       85
       85
       +
                   print_error(f"User '{username}' not found")

     

       86
       86
       +
                   raise typer.Exit(1)

     

       87
       87
       +
               

     

       88
       88
       +
               entries = git_store.list_entries(username, limit)

     

       89
       89
       +
               if not entries:

     

       90
       90
       +
                   print_info(f"No entries found for user '{username}'")

     

       91
       91
       +
                   return

     

       92
       92
       +
               

     

       93
       93
       +
               print_entries_table([entries], [username])

     

       94
       94
       +
           

     

       95
       95
       +
           else:

     

       96
       96
       +
               # List entries for all users

     

       97
       97
       +
               all_entries = []

     

       98
       98
       +
               all_usernames = []

     

       99
       99
       +
               

     

       100
       100
       +
               for user in config.users:

     

       101
       101
       +
                   entries = git_store.list_entries(user.username, limit)

     

       102
       102
       +
                   if entries:

     

       103
       103
       +
                       all_entries.append(entries)

     

       104
       104
       +
                       all_usernames.append(user.username)

     

       105
       105
       +
               

     

       106
       106
       +
               if not all_entries:

     

       107
       107
       +
                   print_info("No entries found")

     

       108
       108
       +
                   return

     

       109
       109
       +
               

     

       110
       110
       +
               print_entries_table(all_entries, all_usernames)

     

       111
       111
       +
       

     

       112
       112
       +
       

     

       113
       113
       +
       def print_entries_table(entries_by_user: list[list], usernames: list[str]) -> None:

     

       114
       114
       +
           """Print a table of entries."""

     

       115
       115
       +
           table = Table(title="Feed Entries")

     

       116
       116
       +
           table.add_column("User", style="cyan", no_wrap=True)

     

       117
       117
       +
           table.add_column("Title", style="bold")

     

       118
       118
       +
           table.add_column("Updated", style="blue")

     

       119
       119
       +
           table.add_column("URL", style="green")

     

       120
       120
       +
           

     

       121
       121
       +
           # Combine all entries with usernames

     

       122
       122
       +
           all_entries = []

     

       123
       123
       +
           for entries, username in zip(entries_by_user, usernames):

     

       124
       124
       +
               for entry in entries:

     

       125
       125
       +
                   all_entries.append((username, entry))

     

       126
       126
       +
           

     

       127
       127
       +
           # Sort by updated time (newest first)

     

       128
       128
       +
           all_entries.sort(key=lambda x: x[1].updated, reverse=True)

     

       129
       129
       +
           

     

       130
       130
       +
           for username, entry in all_entries:

     

       131
       131
       +
               # Format updated time

     

       132
       132
       +
               updated_str = entry.updated.strftime("%Y-%m-%d %H:%M")

     

       133
       133
       +
               

     

       134
       134
       +
               # Truncate title if too long

     

       135
       135
       +
               title = entry.title

     

       136
       136
       +
               if len(title) > 50:

     

       137
       137
       +
                   title = title[:47] + "..."

     

       138
       138
       +
               

     

       139
       139
       +
               table.add_row(

     

       140
       140
       +
                   username,

     

       141
       141
       +
                   title,

     

       142
       142
       +
                   updated_str,

     

       143
       143
       +
                   str(entry.link),

     

       144
       144
       +
               )

     

       145
       145
       +
           

     

       146
       146
       +
           console.print(table)

+142

src/thicket/cli/commands/sync.py

···

       1
       1
       +
       """Sync command for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       import asyncio

     

       4
       4
       +
       from pathlib import Path

     

       5
       5
       +
       from typing import Optional

     

       6
       6
       +
       

     

       7
       7
       +
       import typer

     

       8
       8
       +
       from rich.progress import track

     

       9
       9
       +
       

     

       10
       10
       +
       from ...core.feed_parser import FeedParser

     

       11
       11
       +
       from ...core.git_store import GitStore

     

       12
       12
       +
       from ..main import app

     

       13
       13
       +
       from ..utils import (

     

       14
       14
       +
           create_progress,

     

       15
       15
       +
           load_config,

     

       16
       16
       +
           print_error,

     

       17
       17
       +
           print_info,

     

       18
       18
       +
           print_success,

     

       19
       19
       +
       )

     

       20
       20
       +
       

     

       21
       21
       +
       

     

       22
       22
       +
       @app.command()

     

       23
       23
       +
       def sync(

     

       24
       24
       +
           all_users: bool = typer.Option(

     

       25
       25
       +
               False, "--all", "-a", help="Sync all users and feeds"

     

       26
       26
       +
           ),

     

       27
       27
       +
           user: Optional[str] = typer.Option(

     

       28
       28
       +
               None, "--user", "-u", help="Sync specific user only"

     

       29
       29
       +
           ),

     

       30
       30
       +
           config_file: Optional[Path] = typer.Option(

     

       31
       31
       +
               Path("thicket.yaml"), "--config", help="Configuration file path"

     

       32
       32
       +
           ),

     

       33
       33
       +
           dry_run: bool = typer.Option(

     

       34
       34
       +
               False, "--dry-run", help="Show what would be synced without making changes"

     

       35
       35
       +
           ),

     

       36
       36
       +
       ) -> None:

     

       37
       37
       +
           """Sync feeds and store entries in Git repository."""

     

       38
       38
       +
           

     

       39
       39
       +
           # Load configuration

     

       40
       40
       +
           config = load_config(config_file)

     

       41
       41
       +
           

     

       42
       42
       +
           # Determine which users to sync

     

       43
       43
       +
           users_to_sync = []

     

       44
       44
       +
           if all_users:

     

       45
       45
       +
               users_to_sync = config.users

     

       46
       46
       +
           elif user:

     

       47
       47
       +
               user_config = config.find_user(user)

     

       48
       48
       +
               if not user_config:

     

       49
       49
       +
                   print_error(f"User '{user}' not found")

     

       50
       50
       +
                   raise typer.Exit(1)

     

       51
       51
       +
               users_to_sync = [user_config]

     

       52
       52
       +
           else:

     

       53
       53
       +
               print_error("Specify --all to sync all users or --user to sync a specific user")

     

       54
       54
       +
               raise typer.Exit(1)

     

       55
       55
       +
           

     

       56
       56
       +
           if not users_to_sync:

     

       57
       57
       +
               print_info("No users configured to sync")

     

       58
       58
       +
               return

     

       59
       59
       +
           

     

       60
       60
       +
           # Initialize Git store

     

       61
       61
       +
           git_store = GitStore(config.git_store)

     

       62
       62
       +
           

     

       63
       63
       +
           # Sync each user

     

       64
       64
       +
           total_new_entries = 0

     

       65
       65
       +
           total_updated_entries = 0

     

       66
       66
       +
           

     

       67
       67
       +
           for user_config in users_to_sync:

     

       68
       68
       +
               print_info(f"Syncing user: {user_config.username}")

     

       69
       69
       +
               

     

       70
       70
       +
               user_new_entries = 0

     

       71
       71
       +
               user_updated_entries = 0

     

       72
       72
       +
               

     

       73
       73
       +
               # Sync each feed for the user

     

       74
       74
       +
               for feed_url in track(user_config.feeds, description=f"Syncing {user_config.username}'s feeds"):

     

       75
       75
       +
                   try:

     

       76
       76
       +
                       new_entries, updated_entries = asyncio.run(

     

       77
       77
       +
                           sync_feed(git_store, user_config.username, feed_url, dry_run)

     

       78
       78
       +
                       )

     

       79
       79
       +
                       user_new_entries += new_entries

     

       80
       80
       +
                       user_updated_entries += updated_entries

     

       81
       81
       +
                       

     

       82
       82
       +
                   except Exception as e:

     

       83
       83
       +
                       print_error(f"Failed to sync feed {feed_url}: {e}")

     

       84
       84
       +
                       continue

     

       85
       85
       +
               

     

       86
       86
       +
               print_info(f"User {user_config.username}: {user_new_entries} new, {user_updated_entries} updated")

     

       87
       87
       +
               total_new_entries += user_new_entries

     

       88
       88
       +
               total_updated_entries += user_updated_entries

     

       89
       89
       +
           

     

       90
       90
       +
           # Commit changes if not dry run

     

       91
       91
       +
           if not dry_run and (total_new_entries > 0 or total_updated_entries > 0):

     

       92
       92
       +
               commit_message = f"Sync feeds: {total_new_entries} new entries, {total_updated_entries} updated"

     

       93
       93
       +
               git_store.commit_changes(commit_message)

     

       94
       94
       +
               print_success(f"Committed changes: {commit_message}")

     

       95
       95
       +
           

     

       96
       96
       +
           # Summary

     

       97
       97
       +
           if dry_run:

     

       98
       98
       +
               print_info(f"Dry run complete: would sync {total_new_entries} new entries, {total_updated_entries} updated")

     

       99
       99
       +
           else:

     

       100
       100
       +
               print_success(f"Sync complete: {total_new_entries} new entries, {total_updated_entries} updated")

     

       101
       101
       +
       

     

       102
       102
       +
       

     

       103
       103
       +
       async def sync_feed(git_store: GitStore, username: str, feed_url, dry_run: bool) -> tuple[int, int]:

     

       104
       104
       +
           """Sync a single feed for a user."""

     

       105
       105
       +
           

     

       106
       106
       +
           parser = FeedParser()

     

       107
       107
       +
           

     

       108
       108
       +
           try:

     

       109
       109
       +
               # Fetch and parse feed

     

       110
       110
       +
               content = await parser.fetch_feed(feed_url)

     

       111
       111
       +
               metadata, entries = parser.parse_feed(content, feed_url)

     

       112
       112
       +
               

     

       113
       113
       +
               new_entries = 0

     

       114
       114
       +
               updated_entries = 0

     

       115
       115
       +
               

     

       116
       116
       +
               # Process each entry

     

       117
       117
       +
               for entry in entries:

     

       118
       118
       +
                   try:

     

       119
       119
       +
                       # Check if entry already exists

     

       120
       120
       +
                       existing_entry = git_store.get_entry(username, entry.id)

     

       121
       121
       +
                       

     

       122
       122
       +
                       if existing_entry:

     

       123
       123
       +
                           # Check if entry has been updated

     

       124
       124
       +
                           if existing_entry.updated != entry.updated:

     

       125
       125
       +
                               if not dry_run:

     

       126
       126
       +
                                   git_store.store_entry(username, entry)

     

       127
       127
       +
                               updated_entries += 1

     

       128
       128
       +
                       else:

     

       129
       129
       +
                           # New entry

     

       130
       130
       +
                           if not dry_run:

     

       131
       131
       +
                               git_store.store_entry(username, entry)

     

       132
       132
       +
                           new_entries += 1

     

       133
       133
       +
                           

     

       134
       134
       +
                   except Exception as e:

     

       135
       135
       +
                       print_error(f"Failed to process entry {entry.id}: {e}")

     

       136
       136
       +
                       continue

     

       137
       137
       +
               

     

       138
       138
       +
               return new_entries, updated_entries

     

       139
       139
       +
               

     

       140
       140
       +
           except Exception as e:

     

       141
       141
       +
               print_error(f"Failed to sync feed {feed_url}: {e}")

     

       142
       142
       +
               return 0, 0

+45

src/thicket/cli/main.py

···

       1
       1
       +
       """Main CLI application using Typer."""

     

       2
       2
       +
       

     

       3
       3
       +
       import typer

     

       4
       4
       +
       from rich.console import Console

     

       5
       5
       +
       

     

       6
       6
       +
       from .. import __version__

     

       7
       7
       +
       

     

       8
       8
       +
       app = typer.Typer(

     

       9
       9
       +
           name="thicket",

     

       10
       10
       +
           help="A CLI tool for persisting Atom/RSS feeds in Git repositories",

     

       11
       11
       +
           no_args_is_help=True,

     

       12
       12
       +
           rich_markup_mode="rich",

     

       13
       13
       +
       )

     

       14
       14
       +
       

     

       15
       15
       +
       console = Console()

     

       16
       16
       +
       

     

       17
       17
       +
       

     

       18
       18
       +
       def version_callback(value: bool) -> None:

     

       19
       19
       +
           """Show version and exit."""

     

       20
       20
       +
           if value:

     

       21
       21
       +
               console.print(f"thicket version {__version__}")

     

       22
       22
       +
               raise typer.Exit()

     

       23
       23
       +
       

     

       24
       24
       +
       

     

       25
       25
       +
       @app.callback()

     

       26
       26
       +
       def main(

     

       27
       27
       +
           version: bool = typer.Option(

     

       28
       28
       +
               None,

     

       29
       29
       +
               "--version",

     

       30
       30
       +
               "-v",

     

       31
       31
       +
               help="Show the version and exit",

     

       32
       32
       +
               callback=version_callback,

     

       33
       33
       +
               is_eager=True,

     

       34
       34
       +
           ),

     

       35
       35
       +
       ) -> None:

     

       36
       36
       +
           """Thicket: A CLI tool for persisting Atom/RSS feeds in Git repositories."""

     

       37
       37
       +
           pass

     

       38
       38
       +
       

     

       39
       39
       +
       

     

       40
       40
       +
       # Import commands to register them

     

       41
       41
       +
       from .commands import duplicates, init, add, sync, list_cmd  # noqa: E402

     

       42
       42
       +
       

     

       43
       43
       +
       

     

       44
       44
       +
       if __name__ == "__main__":

     

       45
       45
       +
           app()

+125

src/thicket/cli/utils.py

···

       1
       1
       +
       """CLI utilities and helpers."""

     

       2
       2
       +
       

     

       3
       3
       +
       from pathlib import Path

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       import typer

     

       7
       7
       +
       from rich.console import Console

     

       8
       8
       +
       from rich.progress import Progress, SpinnerColumn, TextColumn

     

       9
       9
       +
       from rich.table import Table

     

       10
       10
       +
       

     

       11
       11
       +
       from ..models import ThicketConfig

     

       12
       12
       +
       

     

       13
       13
       +
       console = Console()

     

       14
       14
       +
       

     

       15
       15
       +
       

     

       16
       16
       +
       def load_config(config_path: Optional[Path] = None) -> ThicketConfig:

     

       17
       17
       +
           """Load thicket configuration from file or environment."""

     

       18
       18
       +
           if config_path and config_path.exists():

     

       19
       19
       +
               import yaml

     

       20
       20
       +
               

     

       21
       21
       +
               with open(config_path) as f:

     

       22
       22
       +
                   config_data = yaml.safe_load(f)

     

       23
       23
       +
               

     

       24
       24
       +
               # Convert to ThicketConfig

     

       25
       25
       +
               return ThicketConfig(**config_data)

     

       26
       26
       +
           

     

       27
       27
       +
           # Try to load from default locations or environment

     

       28
       28
       +
           try:

     

       29
       29
       +
               return ThicketConfig()

     

       30
       30
       +
           except Exception as e:

     

       31
       31
       +
               console.print(f"[red]Error loading configuration: {e}[/red]")

     

       32
       32
       +
               console.print("[yellow]Run 'thicket init' to create a new configuration.[/yellow]")

     

       33
       33
       +
               raise typer.Exit(1)

     

       34
       34
       +
       

     

       35
       35
       +
       

     

       36
       36
       +
       def save_config(config: ThicketConfig, config_path: Path) -> None:

     

       37
       37
       +
           """Save thicket configuration to file."""

     

       38
       38
       +
           import yaml

     

       39
       39
       +
           

     

       40
       40
       +
           config_data = config.model_dump(mode="json")

     

       41
       41
       +
           

     

       42
       42
       +
           # Convert Path objects to strings for YAML serialization

     

       43
       43
       +
           config_data["git_store"] = str(config_data["git_store"])

     

       44
       44
       +
           config_data["cache_dir"] = str(config_data["cache_dir"])

     

       45
       45
       +
           

     

       46
       46
       +
           with open(config_path, "w") as f:

     

       47
       47
       +
               yaml.dump(config_data, f, default_flow_style=False, sort_keys=False)

     

       48
       48
       +
       

     

       49
       49
       +
       

     

       50
       50
       +
       def create_progress() -> Progress:

     

       51
       51
       +
           """Create a Rich progress display."""

     

       52
       52
       +
           return Progress(

     

       53
       53
       +
               SpinnerColumn(),

     

       54
       54
       +
               TextColumn("[progress.description]{task.description}"),

     

       55
       55
       +
               console=console,

     

       56
       56
       +
               transient=True,

     

       57
       57
       +
           )

     

       58
       58
       +
       

     

       59
       59
       +
       

     

       60
       60
       +
       def print_users_table(config: ThicketConfig) -> None:

     

       61
       61
       +
           """Print a table of users and their feeds."""

     

       62
       62
       +
           table = Table(title="Users and Feeds")

     

       63
       63
       +
           table.add_column("Username", style="cyan", no_wrap=True)

     

       64
       64
       +
           table.add_column("Display Name", style="magenta")

     

       65
       65
       +
           table.add_column("Email", style="blue")

     

       66
       66
       +
           table.add_column("Homepage", style="green")

     

       67
       67
       +
           table.add_column("Feeds", style="yellow")

     

       68
       68
       +
           

     

       69
       69
       +
           for user in config.users:

     

       70
       70
       +
               feeds_str = "\n".join(str(feed) for feed in user.feeds)

     

       71
       71
       +
               table.add_row(

     

       72
       72
       +
                   user.username,

     

       73
       73
       +
                   user.display_name or "",

     

       74
       74
       +
                   user.email or "",

     

       75
       75
       +
                   str(user.homepage) if user.homepage else "",

     

       76
       76
       +
                   feeds_str,

     

       77
       77
       +
               )

     

       78
       78
       +
           

     

       79
       79
       +
           console.print(table)

     

       80
       80
       +
       

     

       81
       81
       +
       

     

       82
       82
       +
       def print_feeds_table(config: ThicketConfig, username: Optional[str] = None) -> None:

     

       83
       83
       +
           """Print a table of feeds, optionally filtered by username."""

     

       84
       84
       +
           table = Table(title=f"Feeds{f' for {username}' if username else ''}")

     

       85
       85
       +
           table.add_column("Username", style="cyan", no_wrap=True)

     

       86
       86
       +
           table.add_column("Feed URL", style="blue")

     

       87
       87
       +
           table.add_column("Status", style="green")

     

       88
       88
       +
           

     

       89
       89
       +
           users = [config.find_user(username)] if username else config.users

     

       90
       90
       +
           users = [u for u in users if u is not None]

     

       91
       91
       +
           

     

       92
       92
       +
           for user in users:

     

       93
       93
       +
               for feed in user.feeds:

     

       94
       94
       +
                   table.add_row(

     

       95
       95
       +
                       user.username,

     

       96
       96
       +
                       str(feed),

     

       97
       97
       +
                       "Active",  # TODO: Add actual status checking

     

       98
       98
       +
                   )

     

       99
       99
       +
           

     

       100
       100
       +
           console.print(table)

     

       101
       101
       +
       

     

       102
       102
       +
       

     

       103
       103
       +
       def confirm_action(message: str, default: bool = False) -> bool:

     

       104
       104
       +
           """Prompt for confirmation."""

     

       105
       105
       +
           return typer.confirm(message, default=default)

     

       106
       106
       +
       

     

       107
       107
       +
       

     

       108
       108
       +
       def print_success(message: str) -> None:

     

       109
       109
       +
           """Print a success message."""

     

       110
       110
       +
           console.print(f"[green][/green] {message}")

     

       111
       111
       +
       

     

       112
       112
       +
       

     

       113
       113
       +
       def print_error(message: str) -> None:

     

       114
       114
       +
           """Print an error message."""

     

       115
       115
       +
           console.print(f"[red][/red] {message}")

     

       116
       116
       +
       

     

       117
       117
       +
       

     

       118
       118
       +
       def print_warning(message: str) -> None:

     

       119
       119
       +
           """Print a warning message."""

     

       120
       120
       +
           console.print(f"[yellow]�[/yellow] {message}")

     

       121
       121
       +
       

     

       122
       122
       +
       

     

       123
       123
       +
       def print_info(message: str) -> None:

     

       124
       124
       +
           """Print an info message."""

     

       125
       125
       +
           console.print(f"[blue]9[/blue] {message}")

src/thicket/core/__init__.py

···

       1
       1
       +
       """Core business logic for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from .feed_parser import FeedParser

     

       4
       4
       +
       from .git_store import GitStore

     

       5
       5
       +
       

     

       6
       6
       +
       __all__ = ["FeedParser", "GitStore"]

+262

src/thicket/core/feed_parser.py

···

       1
       1
       +
       """Feed parsing and normalization with auto-discovery."""

     

       2
       2
       +
       

     

       3
       3
       +
       from datetime import datetime

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       from urllib.parse import urljoin, urlparse

     

       6
       6
       +
       

     

       7
       7
       +
       import bleach

     

       8
       8
       +
       import feedparser

     

       9
       9
       +
       import httpx

     

       10
       10
       +
       from pydantic import HttpUrl, ValidationError

     

       11
       11
       +
       

     

       12
       12
       +
       from ..models import AtomEntry, FeedMetadata

     

       13
       13
       +
       

     

       14
       14
       +
       

     

       15
       15
       +
       class FeedParser:

     

       16
       16
       +
           """Parser for RSS/Atom feeds with normalization and auto-discovery."""

     

       17
       17
       +
       

     

       18
       18
       +
           def __init__(self, user_agent: str = "thicket/0.1.0"):

     

       19
       19
       +
               """Initialize the feed parser."""

     

       20
       20
       +
               self.user_agent = user_agent

     

       21
       21
       +
               self.allowed_tags = [

     

       22
       22
       +
                   "a", "abbr", "acronym", "b", "blockquote", "br", "code", "em",

     

       23
       23
       +
                   "i", "li", "ol", "p", "pre", "strong", "ul", "h1", "h2", "h3",

     

       24
       24
       +
                   "h4", "h5", "h6", "img", "div", "span",

     

       25
       25
       +
               ]

     

       26
       26
       +
               self.allowed_attributes = {

     

       27
       27
       +
                   "a": ["href", "title"],

     

       28
       28
       +
                   "abbr": ["title"],

     

       29
       29
       +
                   "acronym": ["title"],

     

       30
       30
       +
                   "img": ["src", "alt", "title", "width", "height"],

     

       31
       31
       +
                   "blockquote": ["cite"],

     

       32
       32
       +
               }

     

       33
       33
       +
       

     

       34
       34
       +
           async def fetch_feed(self, url: HttpUrl) -> str:

     

       35
       35
       +
               """Fetch feed content from URL."""

     

       36
       36
       +
               async with httpx.AsyncClient() as client:

     

       37
       37
       +
                   response = await client.get(

     

       38
       38
       +
                       str(url),

     

       39
       39
       +
                       headers={"User-Agent": self.user_agent},

     

       40
       40
       +
                       timeout=30.0,

     

       41
       41
       +
                       follow_redirects=True,

     

       42
       42
       +
                   )

     

       43
       43
       +
                   response.raise_for_status()

     

       44
       44
       +
                   return response.text

     

       45
       45
       +
       

     

       46
       46
       +
           def parse_feed(self, content: str, source_url: Optional[HttpUrl] = None) -> tuple[FeedMetadata, list[AtomEntry]]:

     

       47
       47
       +
               """Parse feed content and return metadata and entries."""

     

       48
       48
       +
               parsed = feedparser.parse(content)

     

       49
       49
       +
               

     

       50
       50
       +
               if parsed.bozo and parsed.bozo_exception:

     

       51
       51
       +
                   # Try to continue with potentially malformed feed

     

       52
       52
       +
                   pass

     

       53
       53
       +
               

     

       54
       54
       +
               # Extract feed metadata

     

       55
       55
       +
               feed_meta = self._extract_feed_metadata(parsed.feed)

     

       56
       56
       +
               

     

       57
       57
       +
               # Extract and normalize entries

     

       58
       58
       +
               entries = []

     

       59
       59
       +
               for entry in parsed.entries:

     

       60
       60
       +
                   try:

     

       61
       61
       +
                       atom_entry = self._normalize_entry(entry, source_url)

     

       62
       62
       +
                       entries.append(atom_entry)

     

       63
       63
       +
                   except Exception as e:

     

       64
       64
       +
                       # Log error but continue processing other entries

     

       65
       65
       +
                       print(f"Error processing entry {getattr(entry, 'id', 'unknown')}: {e}")

     

       66
       66
       +
                       continue

     

       67
       67
       +
               

     

       68
       68
       +
               return feed_meta, entries

     

       69
       69
       +
       

     

       70
       70
       +
           def _extract_feed_metadata(self, feed: feedparser.FeedParserDict) -> FeedMetadata:

     

       71
       71
       +
               """Extract metadata from feed for auto-discovery."""

     

       72
       72
       +
               # Parse author information

     

       73
       73
       +
               author_name = None

     

       74
       74
       +
               author_email = None

     

       75
       75
       +
               author_uri = None

     

       76
       76
       +
               

     

       77
       77
       +
               if hasattr(feed, 'author_detail'):

     

       78
       78
       +
                   author_name = feed.author_detail.get('name')

     

       79
       79
       +
                   author_email = feed.author_detail.get('email')

     

       80
       80
       +
                   author_uri = feed.author_detail.get('href')

     

       81
       81
       +
               elif hasattr(feed, 'author'):

     

       82
       82
       +
                   author_name = feed.author

     

       83
       83
       +
               

     

       84
       84
       +
               # Parse managing editor for RSS feeds

     

       85
       85
       +
               if not author_email and hasattr(feed, 'managingEditor'):

     

       86
       86
       +
                   author_email = feed.managingEditor

     

       87
       87
       +
               

     

       88
       88
       +
               # Parse feed link

     

       89
       89
       +
               feed_link = None

     

       90
       90
       +
               if hasattr(feed, 'link'):

     

       91
       91
       +
                   try:

     

       92
       92
       +
                       feed_link = HttpUrl(feed.link)

     

       93
       93
       +
                   except ValidationError:

     

       94
       94
       +
                       pass

     

       95
       95
       +
               

     

       96
       96
       +
               # Parse image/icon/logo

     

       97
       97
       +
               logo = None

     

       98
       98
       +
               icon = None

     

       99
       99
       +
               image_url = None

     

       100
       100
       +
               

     

       101
       101
       +
               if hasattr(feed, 'image'):

     

       102
       102
       +
                   try:

     

       103
       103
       +
                       image_url = HttpUrl(feed.image.get('href', feed.image.get('url', '')))

     

       104
       104
       +
                   except (ValidationError, AttributeError):

     

       105
       105
       +
                       pass

     

       106
       106
       +
               

     

       107
       107
       +
               if hasattr(feed, 'icon'):

     

       108
       108
       +
                   try:

     

       109
       109
       +
                       icon = HttpUrl(feed.icon)

     

       110
       110
       +
                   except ValidationError:

     

       111
       111
       +
                       pass

     

       112
       112
       +
               

     

       113
       113
       +
               if hasattr(feed, 'logo'):

     

       114
       114
       +
                   try:

     

       115
       115
       +
                       logo = HttpUrl(feed.logo)

     

       116
       116
       +
                   except ValidationError:

     

       117
       117
       +
                       pass

     

       118
       118
       +
               

     

       119
       119
       +
               return FeedMetadata(

     

       120
       120
       +
                   title=getattr(feed, 'title', None),

     

       121
       121
       +
                   author_name=author_name,

     

       122
       122
       +
                   author_email=author_email,

     

       123
       123
       +
                   author_uri=HttpUrl(author_uri) if author_uri else None,

     

       124
       124
       +
                   link=feed_link,

     

       125
       125
       +
                   logo=logo,

     

       126
       126
       +
                   icon=icon,

     

       127
       127
       +
                   image_url=image_url,

     

       128
       128
       +
                   description=getattr(feed, 'description', None),

     

       129
       129
       +
               )

     

       130
       130
       +
       

     

       131
       131
       +
           def _normalize_entry(self, entry: feedparser.FeedParserDict, source_url: Optional[HttpUrl] = None) -> AtomEntry:

     

       132
       132
       +
               """Normalize an entry to Atom format."""

     

       133
       133
       +
               # Parse timestamps

     

       134
       134
       +
               updated = self._parse_timestamp(entry.get('updated_parsed') or entry.get('published_parsed'))

     

       135
       135
       +
               published = self._parse_timestamp(entry.get('published_parsed'))

     

       136
       136
       +
               

     

       137
       137
       +
               # Parse content

     

       138
       138
       +
               content = self._extract_content(entry)

     

       139
       139
       +
               content_type = self._extract_content_type(entry)

     

       140
       140
       +
               

     

       141
       141
       +
               # Parse author

     

       142
       142
       +
               author = self._extract_author(entry)

     

       143
       143
       +
               

     

       144
       144
       +
               # Parse categories/tags

     

       145
       145
       +
               categories = []

     

       146
       146
       +
               if hasattr(entry, 'tags'):

     

       147
       147
       +
                   categories = [tag.get('term', '') for tag in entry.tags if tag.get('term')]

     

       148
       148
       +
               

     

       149
       149
       +
               # Sanitize HTML content

     

       150
       150
       +
               if content:

     

       151
       151
       +
                   content = self._sanitize_html(content)

     

       152
       152
       +
               

     

       153
       153
       +
               summary = entry.get('summary', '')

     

       154
       154
       +
               if summary:

     

       155
       155
       +
                   summary = self._sanitize_html(summary)

     

       156
       156
       +
               

     

       157
       157
       +
               return AtomEntry(

     

       158
       158
       +
                   id=entry.get('id', entry.get('link', '')),

     

       159
       159
       +
                   title=entry.get('title', ''),

     

       160
       160
       +
                   link=HttpUrl(entry.get('link', '')),

     

       161
       161
       +
                   updated=updated,

     

       162
       162
       +
                   published=published,

     

       163
       163
       +
                   summary=summary or None,

     

       164
       164
       +
                   content=content or None,

     

       165
       165
       +
                   content_type=content_type,

     

       166
       166
       +
                   author=author,

     

       167
       167
       +
                   categories=categories,

     

       168
       168
       +
                   rights=entry.get('rights', None),

     

       169
       169
       +
                   source=str(source_url) if source_url else None,

     

       170
       170
       +
               )

     

       171
       171
       +
       

     

       172
       172
       +
           def _parse_timestamp(self, time_struct) -> datetime:

     

       173
       173
       +
               """Parse feedparser time struct to datetime."""

     

       174
       174
       +
               if time_struct:

     

       175
       175
       +
                   return datetime(*time_struct[:6])

     

       176
       176
       +
               return datetime.now()

     

       177
       177
       +
       

     

       178
       178
       +
           def _extract_content(self, entry: feedparser.FeedParserDict) -> Optional[str]:

     

       179
       179
       +
               """Extract the best content from an entry."""

     

       180
       180
       +
               # Prefer content over summary

     

       181
       181
       +
               if hasattr(entry, 'content') and entry.content:

     

       182
       182
       +
                   # Find the best content (prefer text/html, then text/plain)

     

       183
       183
       +
                   for content_item in entry.content:

     

       184
       184
       +
                       if content_item.get('type') in ['text/html', 'html']:

     

       185
       185
       +
                           return content_item.get('value', '')

     

       186
       186
       +
                       elif content_item.get('type') in ['text/plain', 'text']:

     

       187
       187
       +
                           return content_item.get('value', '')

     

       188
       188
       +
                   # Fallback to first content item

     

       189
       189
       +
                   return entry.content[0].get('value', '')

     

       190
       190
       +
               

     

       191
       191
       +
               # Fallback to summary

     

       192
       192
       +
               return entry.get('summary', '')

     

       193
       193
       +
       

     

       194
       194
       +
           def _extract_content_type(self, entry: feedparser.FeedParserDict) -> str:

     

       195
       195
       +
               """Extract content type from entry."""

     

       196
       196
       +
               if hasattr(entry, 'content') and entry.content:

     

       197
       197
       +
                   content_type = entry.content[0].get('type', 'html')

     

       198
       198
       +
                   # Normalize content type

     

       199
       199
       +
                   if content_type in ['text/html', 'html']:

     

       200
       200
       +
                       return 'html'

     

       201
       201
       +
                   elif content_type in ['text/plain', 'text']:

     

       202
       202
       +
                       return 'text'

     

       203
       203
       +
                   elif content_type == 'xhtml':

     

       204
       204
       +
                       return 'xhtml'

     

       205
       205
       +
               return 'html'

     

       206
       206
       +
       

     

       207
       207
       +
           def _extract_author(self, entry: feedparser.FeedParserDict) -> Optional[dict]:

     

       208
       208
       +
               """Extract author information from entry."""

     

       209
       209
       +
               author = {}

     

       210
       210
       +
               

     

       211
       211
       +
               if hasattr(entry, 'author_detail'):

     

       212
       212
       +
                   author.update({

     

       213
       213
       +
                       'name': entry.author_detail.get('name'),

     

       214
       214
       +
                       'email': entry.author_detail.get('email'),

     

       215
       215
       +
                       'uri': entry.author_detail.get('href'),

     

       216
       216
       +
                   })

     

       217
       217
       +
               elif hasattr(entry, 'author'):

     

       218
       218
       +
                   author['name'] = entry.author

     

       219
       219
       +
               

     

       220
       220
       +
               return author if author else None

     

       221
       221
       +
       

     

       222
       222
       +
           def _sanitize_html(self, html: str) -> str:

     

       223
       223
       +
               """Sanitize HTML content to prevent XSS."""

     

       224
       224
       +
               return bleach.clean(

     

       225
       225
       +
                   html,

     

       226
       226
       +
                   tags=self.allowed_tags,

     

       227
       227
       +
                   attributes=self.allowed_attributes,

     

       228
       228
       +
                   strip=True,

     

       229
       229
       +
               )

     

       230
       230
       +
       

     

       231
       231
       +
           def sanitize_entry_id(self, entry_id: str) -> str:

     

       232
       232
       +
               """Sanitize entry ID to be a safe filename."""

     

       233
       233
       +
               # Parse URL to get meaningful parts

     

       234
       234
       +
               parsed = urlparse(entry_id)

     

       235
       235
       +
               

     

       236
       236
       +
               # Start with the path component

     

       237
       237
       +
               if parsed.path:

     

       238
       238
       +
                   # Remove leading slash and replace problematic characters

     

       239
       239
       +
                   safe_id = parsed.path.lstrip('/').replace('/', '_').replace('\\', '_')

     

       240
       240
       +
               else:

     

       241
       241
       +
                   # Use the entire ID as fallback

     

       242
       242
       +
                   safe_id = entry_id

     

       243
       243
       +
               

     

       244
       244
       +
               # Replace problematic characters

     

       245
       245
       +
               safe_chars = []

     

       246
       246
       +
               for char in safe_id:

     

       247
       247
       +
                   if char.isalnum() or char in '-_.':

     

       248
       248
       +
                       safe_chars.append(char)

     

       249
       249
       +
                   else:

     

       250
       250
       +
                       safe_chars.append('_')

     

       251
       251
       +
               

     

       252
       252
       +
               safe_id = ''.join(safe_chars)

     

       253
       253
       +
               

     

       254
       254
       +
               # Ensure it's not too long (max 200 chars)

     

       255
       255
       +
               if len(safe_id) > 200:

     

       256
       256
       +
                   safe_id = safe_id[:200]

     

       257
       257
       +
               

     

       258
       258
       +
               # Ensure it's not empty

     

       259
       259
       +
               if not safe_id:

     

       260
       260
       +
                   safe_id = "entry"

     

       261
       261
       +
               

     

       262
       262
       +
               return safe_id

+321

src/thicket/core/git_store.py

···

       1
       1
       +
       """Git repository operations for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       import json

     

       4
       4
       +
       from datetime import datetime

     

       5
       5
       +
       from pathlib import Path

     

       6
       6
       +
       from typing import Optional

     

       7
       7
       +
       

     

       8
       8
       +
       import git

     

       9
       9
       +
       from git import Repo

     

       10
       10
       +
       

     

       11
       11
       +
       from ..models import AtomEntry, DuplicateMap, GitStoreIndex, UserMetadata

     

       12
       12
       +
       

     

       13
       13
       +
       

     

       14
       14
       +
       class GitStore:

     

       15
       15
       +
           """Manages the Git repository for storing feed entries."""

     

       16
       16
       +
       

     

       17
       17
       +
           def __init__(self, repo_path: Path):

     

       18
       18
       +
               """Initialize the Git store."""

     

       19
       19
       +
               self.repo_path = repo_path

     

       20
       20
       +
               self.repo: Optional[Repo] = None

     

       21
       21
       +
               self._ensure_repo()

     

       22
       22
       +
       

     

       23
       23
       +
           def _ensure_repo(self) -> None:

     

       24
       24
       +
               """Ensure the Git repository exists and is initialized."""

     

       25
       25
       +
               if not self.repo_path.exists():

     

       26
       26
       +
                   self.repo_path.mkdir(parents=True, exist_ok=True)

     

       27
       27
       +
               

     

       28
       28
       +
               try:

     

       29
       29
       +
                   self.repo = Repo(self.repo_path)

     

       30
       30
       +
               except git.InvalidGitRepositoryError:

     

       31
       31
       +
                   # Initialize new repository

     

       32
       32
       +
                   self.repo = Repo.init(self.repo_path)

     

       33
       33
       +
                   self._create_initial_structure()

     

       34
       34
       +
       

     

       35
       35
       +
           def _create_initial_structure(self) -> None:

     

       36
       36
       +
               """Create initial Git store structure."""

     

       37
       37
       +
               # Create index.json

     

       38
       38
       +
               index = GitStoreIndex(

     

       39
       39
       +
                   created=datetime.now(),

     

       40
       40
       +
                   last_updated=datetime.now(),

     

       41
       41
       +
               )

     

       42
       42
       +
               self._save_index(index)

     

       43
       43
       +
               

     

       44
       44
       +
               # Create duplicates.json

     

       45
       45
       +
               duplicates = DuplicateMap()

     

       46
       46
       +
               self._save_duplicates(duplicates)

     

       47
       47
       +
               

     

       48
       48
       +
               # Create initial commit

     

       49
       49
       +
               self.repo.index.add(["index.json", "duplicates.json"])

     

       50
       50
       +
               self.repo.index.commit("Initial thicket repository structure")

     

       51
       51
       +
       

     

       52
       52
       +
           def _save_index(self, index: GitStoreIndex) -> None:

     

       53
       53
       +
               """Save the index to index.json."""

     

       54
       54
       +
               index_path = self.repo_path / "index.json"

     

       55
       55
       +
               with open(index_path, "w") as f:

     

       56
       56
       +
                   json.dump(index.model_dump(mode="json"), f, indent=2, default=str)

     

       57
       57
       +
       

     

       58
       58
       +
           def _load_index(self) -> GitStoreIndex:

     

       59
       59
       +
               """Load the index from index.json."""

     

       60
       60
       +
               index_path = self.repo_path / "index.json"

     

       61
       61
       +
               if not index_path.exists():

     

       62
       62
       +
                   return GitStoreIndex(

     

       63
       63
       +
                       created=datetime.now(),

     

       64
       64
       +
                       last_updated=datetime.now(),

     

       65
       65
       +
                   )

     

       66
       66
       +
               

     

       67
       67
       +
               with open(index_path) as f:

     

       68
       68
       +
                   data = json.load(f)

     

       69
       69
       +
               

     

       70
       70
       +
               return GitStoreIndex(**data)

     

       71
       71
       +
       

     

       72
       72
       +
           def _save_duplicates(self, duplicates: DuplicateMap) -> None:

     

       73
       73
       +
               """Save duplicates map to duplicates.json."""

     

       74
       74
       +
               duplicates_path = self.repo_path / "duplicates.json"

     

       75
       75
       +
               with open(duplicates_path, "w") as f:

     

       76
       76
       +
                   json.dump(duplicates.model_dump(), f, indent=2)

     

       77
       77
       +
       

     

       78
       78
       +
           def _load_duplicates(self) -> DuplicateMap:

     

       79
       79
       +
               """Load duplicates map from duplicates.json."""

     

       80
       80
       +
               duplicates_path = self.repo_path / "duplicates.json"

     

       81
       81
       +
               if not duplicates_path.exists():

     

       82
       82
       +
                   return DuplicateMap()

     

       83
       83
       +
               

     

       84
       84
       +
               with open(duplicates_path) as f:

     

       85
       85
       +
                   data = json.load(f)

     

       86
       86
       +
               

     

       87
       87
       +
               return DuplicateMap(**data)

     

       88
       88
       +
       

     

       89
       89
       +
           def add_user(self, username: str, display_name: Optional[str] = None,

     

       90
       90
       +
                        email: Optional[str] = None, homepage: Optional[str] = None,

     

       91
       91
       +
                        icon: Optional[str] = None, feeds: Optional[list[str]] = None) -> UserMetadata:

     

       92
       92
       +
               """Add a new user to the Git store."""

     

       93
       93
       +
               index = self._load_index()

     

       94
       94
       +
               

     

       95
       95
       +
               # Create user directory

     

       96
       96
       +
               user_dir = self.repo_path / username

     

       97
       97
       +
               user_dir.mkdir(exist_ok=True)

     

       98
       98
       +
               

     

       99
       99
       +
               # Create user metadata

     

       100
       100
       +
               user_metadata = UserMetadata(

     

       101
       101
       +
                   username=username,

     

       102
       102
       +
                   display_name=display_name,

     

       103
       103
       +
                   email=email,

     

       104
       104
       +
                   homepage=homepage,

     

       105
       105
       +
                   icon=icon,

     

       106
       106
       +
                   feeds=feeds or [],

     

       107
       107
       +
                   directory=username,

     

       108
       108
       +
                   created=datetime.now(),

     

       109
       109
       +
                   last_updated=datetime.now(),

     

       110
       110
       +
               )

     

       111
       111
       +
               

     

       112
       112
       +
               # Save user metadata

     

       113
       113
       +
               metadata_path = user_dir / "metadata.json"

     

       114
       114
       +
               with open(metadata_path, "w") as f:

     

       115
       115
       +
                   json.dump(user_metadata.model_dump(mode="json"), f, indent=2, default=str)

     

       116
       116
       +
               

     

       117
       117
       +
               # Update index

     

       118
       118
       +
               index.add_user(user_metadata)

     

       119
       119
       +
               self._save_index(index)

     

       120
       120
       +
               

     

       121
       121
       +
               return user_metadata

     

       122
       122
       +
       

     

       123
       123
       +
           def get_user(self, username: str) -> Optional[UserMetadata]:

     

       124
       124
       +
               """Get user metadata by username."""

     

       125
       125
       +
               index = self._load_index()

     

       126
       126
       +
               return index.get_user(username)

     

       127
       127
       +
       

     

       128
       128
       +
           def update_user(self, username: str, **kwargs) -> bool:

     

       129
       129
       +
               """Update user metadata."""

     

       130
       130
       +
               index = self._load_index()

     

       131
       131
       +
               user = index.get_user(username)

     

       132
       132
       +
               

     

       133
       133
       +
               if not user:

     

       134
       134
       +
                   return False

     

       135
       135
       +
               

     

       136
       136
       +
               # Update user metadata

     

       137
       137
       +
               for key, value in kwargs.items():

     

       138
       138
       +
                   if hasattr(user, key) and value is not None:

     

       139
       139
       +
                       setattr(user, key, value)

     

       140
       140
       +
               

     

       141
       141
       +
               user.update_timestamp()

     

       142
       142
       +
               

     

       143
       143
       +
               # Save user metadata

     

       144
       144
       +
               user_dir = self.repo_path / user.directory

     

       145
       145
       +
               metadata_path = user_dir / "metadata.json"

     

       146
       146
       +
               with open(metadata_path, "w") as f:

     

       147
       147
       +
                   json.dump(user.model_dump(mode="json"), f, indent=2, default=str)

     

       148
       148
       +
               

     

       149
       149
       +
               # Update index

     

       150
       150
       +
               index.add_user(user)

     

       151
       151
       +
               self._save_index(index)

     

       152
       152
       +
               

     

       153
       153
       +
               return True

     

       154
       154
       +
       

     

       155
       155
       +
           def store_entry(self, username: str, entry: AtomEntry) -> bool:

     

       156
       156
       +
               """Store an entry in the user's directory."""

     

       157
       157
       +
               user = self.get_user(username)

     

       158
       158
       +
               if not user:

     

       159
       159
       +
                   return False

     

       160
       160
       +
               

     

       161
       161
       +
               # Sanitize entry ID for filename

     

       162
       162
       +
               from .feed_parser import FeedParser

     

       163
       163
       +
               parser = FeedParser()

     

       164
       164
       +
               safe_id = parser.sanitize_entry_id(entry.id)

     

       165
       165
       +
               

     

       166
       166
       +
               # Create entry file

     

       167
       167
       +
               user_dir = self.repo_path / user.directory

     

       168
       168
       +
               entry_path = user_dir / f"{safe_id}.json"

     

       169
       169
       +
               

     

       170
       170
       +
               # Check if entry already exists

     

       171
       171
       +
               entry_exists = entry_path.exists()

     

       172
       172
       +
               

     

       173
       173
       +
               # Save entry

     

       174
       174
       +
               with open(entry_path, "w") as f:

     

       175
       175
       +
                   json.dump(entry.model_dump(mode="json"), f, indent=2, default=str)

     

       176
       176
       +
               

     

       177
       177
       +
               # Update user metadata if new entry

     

       178
       178
       +
               if not entry_exists:

     

       179
       179
       +
                   user.increment_entry_count()

     

       180
       180
       +
                   self.update_user(username, entry_count=user.entry_count)

     

       181
       181
       +
               

     

       182
       182
       +
               return True

     

       183
       183
       +
       

     

       184
       184
       +
           def get_entry(self, username: str, entry_id: str) -> Optional[AtomEntry]:

     

       185
       185
       +
               """Get an entry by username and entry ID."""

     

       186
       186
       +
               user = self.get_user(username)

     

       187
       187
       +
               if not user:

     

       188
       188
       +
                   return None

     

       189
       189
       +
               

     

       190
       190
       +
               # Sanitize entry ID

     

       191
       191
       +
               from .feed_parser import FeedParser

     

       192
       192
       +
               parser = FeedParser()

     

       193
       193
       +
               safe_id = parser.sanitize_entry_id(entry_id)

     

       194
       194
       +
               

     

       195
       195
       +
               entry_path = self.repo_path / user.directory / f"{safe_id}.json"

     

       196
       196
       +
               if not entry_path.exists():

     

       197
       197
       +
                   return None

     

       198
       198
       +
               

     

       199
       199
       +
               with open(entry_path) as f:

     

       200
       200
       +
                   data = json.load(f)

     

       201
       201
       +
               

     

       202
       202
       +
               return AtomEntry(**data)

     

       203
       203
       +
       

     

       204
       204
       +
           def list_entries(self, username: str, limit: Optional[int] = None) -> list[AtomEntry]:

     

       205
       205
       +
               """List entries for a user."""

     

       206
       206
       +
               user = self.get_user(username)

     

       207
       207
       +
               if not user:

     

       208
       208
       +
                   return []

     

       209
       209
       +
               

     

       210
       210
       +
               user_dir = self.repo_path / user.directory

     

       211
       211
       +
               if not user_dir.exists():

     

       212
       212
       +
                   return []

     

       213
       213
       +
               

     

       214
       214
       +
               entries = []

     

       215
       215
       +
               entry_files = sorted(user_dir.glob("*.json"), key=lambda p: p.stat().st_mtime, reverse=True)

     

       216
       216
       +
               

     

       217
       217
       +
               # Filter out metadata.json

     

       218
       218
       +
               entry_files = [f for f in entry_files if f.name != "metadata.json"]

     

       219
       219
       +
               

     

       220
       220
       +
               if limit:

     

       221
       221
       +
                   entry_files = entry_files[:limit]

     

       222
       222
       +
               

     

       223
       223
       +
               for entry_file in entry_files:

     

       224
       224
       +
                   try:

     

       225
       225
       +
                       with open(entry_file) as f:

     

       226
       226
       +
                           data = json.load(f)

     

       227
       227
       +
                       entries.append(AtomEntry(**data))

     

       228
       228
       +
                   except Exception:

     

       229
       229
       +
                       # Skip invalid entries

     

       230
       230
       +
                       continue

     

       231
       231
       +
               

     

       232
       232
       +
               return entries

     

       233
       233
       +
       

     

       234
       234
       +
           def get_duplicates(self) -> DuplicateMap:

     

       235
       235
       +
               """Get the duplicates map."""

     

       236
       236
       +
               return self._load_duplicates()

     

       237
       237
       +
       

     

       238
       238
       +
           def add_duplicate(self, duplicate_id: str, canonical_id: str) -> None:

     

       239
       239
       +
               """Add a duplicate mapping."""

     

       240
       240
       +
               duplicates = self._load_duplicates()

     

       241
       241
       +
               duplicates.add_duplicate(duplicate_id, canonical_id)

     

       242
       242
       +
               self._save_duplicates(duplicates)

     

       243
       243
       +
       

     

       244
       244
       +
           def remove_duplicate(self, duplicate_id: str) -> bool:

     

       245
       245
       +
               """Remove a duplicate mapping."""

     

       246
       246
       +
               duplicates = self._load_duplicates()

     

       247
       247
       +
               result = duplicates.remove_duplicate(duplicate_id)

     

       248
       248
       +
               self._save_duplicates(duplicates)

     

       249
       249
       +
               return result

     

       250
       250
       +
       

     

       251
       251
       +
           def commit_changes(self, message: str) -> None:

     

       252
       252
       +
               """Commit all changes to the Git repository."""

     

       253
       253
       +
               if not self.repo:

     

       254
       254
       +
                   return

     

       255
       255
       +
               

     

       256
       256
       +
               # Add all changes

     

       257
       257
       +
               self.repo.git.add(A=True)

     

       258
       258
       +
               

     

       259
       259
       +
               # Check if there are changes to commit

     

       260
       260
       +
               if self.repo.index.diff("HEAD"):

     

       261
       261
       +
                   self.repo.index.commit(message)

     

       262
       262
       +
       

     

       263
       263
       +
           def get_stats(self) -> dict:

     

       264
       264
       +
               """Get statistics about the Git store."""

     

       265
       265
       +
               index = self._load_index()

     

       266
       266
       +
               duplicates = self._load_duplicates()

     

       267
       267
       +
               

     

       268
       268
       +
               return {

     

       269
       269
       +
                   "total_users": len(index.users),

     

       270
       270
       +
                   "total_entries": index.total_entries,

     

       271
       271
       +
                   "total_duplicates": len(duplicates.duplicates),

     

       272
       272
       +
                   "last_updated": index.last_updated,

     

       273
       273
       +
                   "repository_size": sum(f.stat().st_size for f in self.repo_path.rglob("*") if f.is_file()),

     

       274
       274
       +
               }

     

       275
       275
       +
       

     

       276
       276
       +
           def search_entries(self, query: str, username: Optional[str] = None,

     

       277
       277
       +
                             limit: Optional[int] = None) -> list[tuple[str, AtomEntry]]:

     

       278
       278
       +
               """Search entries by content."""

     

       279
       279
       +
               results = []

     

       280
       280
       +
               

     

       281
       281
       +
               # Get users to search

     

       282
       282
       +
               index = self._load_index()

     

       283
       283
       +
               users = [index.get_user(username)] if username else list(index.users.values())

     

       284
       284
       +
               users = [u for u in users if u is not None]

     

       285
       285
       +
               

     

       286
       286
       +
               for user in users:

     

       287
       287
       +
                   user_dir = self.repo_path / user.directory

     

       288
       288
       +
                   if not user_dir.exists():

     

       289
       289
       +
                       continue

     

       290
       290
       +
                   

     

       291
       291
       +
                   entry_files = user_dir.glob("*.json")

     

       292
       292
       +
                   entry_files = [f for f in entry_files if f.name != "metadata.json"]

     

       293
       293
       +
                   

     

       294
       294
       +
                   for entry_file in entry_files:

     

       295
       295
       +
                       try:

     

       296
       296
       +
                           with open(entry_file) as f:

     

       297
       297
       +
                               data = json.load(f)

     

       298
       298
       +
                           

     

       299
       299
       +
                           entry = AtomEntry(**data)

     

       300
       300
       +
                           

     

       301
       301
       +
                           # Simple text search in title, summary, and content

     

       302
       302
       +
                           searchable_text = " ".join(filter(None, [

     

       303
       303
       +
                               entry.title,

     

       304
       304
       +
                               entry.summary or "",

     

       305
       305
       +
                               entry.content or "",

     

       306
       306
       +
                           ])).lower()

     

       307
       307
       +
                           

     

       308
       308
       +
                           if query.lower() in searchable_text:

     

       309
       309
       +
                               results.append((user.username, entry))

     

       310
       310
       +
                               

     

       311
       311
       +
                               if limit and len(results) >= limit:

     

       312
       312
       +
                                   return results

     

       313
       313
       +
                           

     

       314
       314
       +
                       except Exception:

     

       315
       315
       +
                           # Skip invalid entries

     

       316
       316
       +
                           continue

     

       317
       317
       +
               

     

       318
       318
       +
               # Sort by updated time (newest first)

     

       319
       319
       +
               results.sort(key=lambda x: x[1].updated, reverse=True)

     

       320
       320
       +
               

     

       321
       321
       +
               return results[:limit] if limit else results

+15

src/thicket/models/__init__.py

···

       1
       1
       +
       """Data models for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from .config import ThicketConfig, UserConfig

     

       4
       4
       +
       from .feed import AtomEntry, DuplicateMap, FeedMetadata

     

       5
       5
       +
       from .user import GitStoreIndex, UserMetadata

     

       6
       6
       +
       

     

       7
       7
       +
       __all__ = [

     

       8
       8
       +
           "ThicketConfig",

     

       9
       9
       +
           "UserConfig",

     

       10
       10
       +
           "AtomEntry",

     

       11
       11
       +
           "DuplicateMap",

     

       12
       12
       +
           "FeedMetadata",

     

       13
       13
       +
           "GitStoreIndex",

     

       14
       14
       +
           "UserMetadata",

     

       15
       15
       +
       ]

+71

src/thicket/models/config.py

···

       1
       1
       +
       """Configuration models for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from pathlib import Path

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       from pydantic import BaseModel, EmailStr, HttpUrl

     

       7
       7
       +
       from pydantic_settings import BaseSettings, SettingsConfigDict

     

       8
       8
       +
       

     

       9
       9
       +
       

     

       10
       10
       +
       class UserConfig(BaseModel):

     

       11
       11
       +
           """Configuration for a single user and their feeds."""

     

       12
       12
       +
       

     

       13
       13
       +
           username: str

     

       14
       14
       +
           feeds: list[HttpUrl]

     

       15
       15
       +
           email: Optional[EmailStr] = None

     

       16
       16
       +
           homepage: Optional[HttpUrl] = None

     

       17
       17
       +
           icon: Optional[HttpUrl] = None

     

       18
       18
       +
           display_name: Optional[str] = None

     

       19
       19
       +
       

     

       20
       20
       +
       

     

       21
       21
       +
       class ThicketConfig(BaseSettings):

     

       22
       22
       +
           """Main configuration for thicket."""

     

       23
       23
       +
       

     

       24
       24
       +
           model_config = SettingsConfigDict(

     

       25
       25
       +
               env_prefix="THICKET_",

     

       26
       26
       +
               env_file=".env",

     

       27
       27
       +
               yaml_file="thicket.yaml",

     

       28
       28
       +
               case_sensitive=False,

     

       29
       29
       +
           )

     

       30
       30
       +
       

     

       31
       31
       +
           git_store: Path

     

       32
       32
       +
           cache_dir: Path

     

       33
       33
       +
           users: list[UserConfig] = []

     

       34
       34
       +
       

     

       35
       35
       +
           def find_user(self, username: str) -> Optional[UserConfig]:

     

       36
       36
       +
               """Find a user by username."""

     

       37
       37
       +
               for user in self.users:

     

       38
       38
       +
                   if user.username == username:

     

       39
       39
       +
                       return user

     

       40
       40
       +
               return None

     

       41
       41
       +
       

     

       42
       42
       +
           def add_user(self, user: UserConfig) -> None:

     

       43
       43
       +
               """Add a new user or update existing user."""

     

       44
       44
       +
               existing = self.find_user(user.username)

     

       45
       45
       +
               if existing:

     

       46
       46
       +
                   # Update existing user

     

       47
       47
       +
                   existing.feeds = list(set(existing.feeds + user.feeds))

     

       48
       48
       +
                   existing.email = user.email or existing.email

     

       49
       49
       +
                   existing.homepage = user.homepage or existing.homepage

     

       50
       50
       +
                   existing.icon = user.icon or existing.icon

     

       51
       51
       +
                   existing.display_name = user.display_name or existing.display_name

     

       52
       52
       +
               else:

     

       53
       53
       +
                   # Add new user

     

       54
       54
       +
                   self.users.append(user)

     

       55
       55
       +
       

     

       56
       56
       +
           def remove_user(self, username: str) -> bool:

     

       57
       57
       +
               """Remove a user by username. Returns True if user was found and removed."""

     

       58
       58
       +
               for i, user in enumerate(self.users):

     

       59
       59
       +
                   if user.username == username:

     

       60
       60
       +
                       del self.users[i]

     

       61
       61
       +
                       return True

     

       62
       62
       +
               return False

     

       63
       63
       +
       

     

       64
       64
       +
           def add_feed_to_user(self, username: str, feed_url: HttpUrl) -> bool:

     

       65
       65
       +
               """Add a feed to an existing user. Returns True if user was found."""

     

       66
       66
       +
               user = self.find_user(username)

     

       67
       67
       +
               if user:

     

       68
       68
       +
                   if feed_url not in user.feeds:

     

       69
       69
       +
                       user.feeds.append(feed_url)

     

       70
       70
       +
                   return True

     

       71
       71
       +
               return False

+86

src/thicket/models/feed.py

···

       1
       1
       +
       """Feed and entry models for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from datetime import datetime

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       from pydantic import BaseModel, ConfigDict, EmailStr, HttpUrl

     

       7
       7
       +
       

     

       8
       8
       +
       

     

       9
       9
       +
       class AtomEntry(BaseModel):

     

       10
       10
       +
           """Represents an Atom feed entry stored in the Git repository."""

     

       11
       11
       +
       

     

       12
       12
       +
           model_config = ConfigDict(

     

       13
       13
       +
               json_encoders={datetime: lambda v: v.isoformat()},

     

       14
       14
       +
               str_strip_whitespace=True,

     

       15
       15
       +
           )

     

       16
       16
       +
       

     

       17
       17
       +
           id: str  # Original Atom ID

     

       18
       18
       +
           title: str

     

       19
       19
       +
           link: HttpUrl

     

       20
       20
       +
           updated: datetime

     

       21
       21
       +
           published: Optional[datetime] = None

     

       22
       22
       +
           summary: Optional[str] = None

     

       23
       23
       +
           content: Optional[str] = None  # Full body content from Atom entry

     

       24
       24
       +
           content_type: Optional[str] = "html"  # text, html, xhtml

     

       25
       25
       +
           author: Optional[dict] = None

     

       26
       26
       +
           categories: list[str] = []

     

       27
       27
       +
           rights: Optional[str] = None  # Copyright info

     

       28
       28
       +
           source: Optional[str] = None  # Source feed URL

     

       29
       29
       +
       

     

       30
       30
       +
       

     

       31
       31
       +
       class FeedMetadata(BaseModel):

     

       32
       32
       +
           """Metadata extracted from a feed for auto-discovery."""

     

       33
       33
       +
       

     

       34
       34
       +
           title: Optional[str] = None

     

       35
       35
       +
           author_name: Optional[str] = None

     

       36
       36
       +
           author_email: Optional[EmailStr] = None

     

       37
       37
       +
           author_uri: Optional[HttpUrl] = None

     

       38
       38
       +
           link: Optional[HttpUrl] = None

     

       39
       39
       +
           logo: Optional[HttpUrl] = None

     

       40
       40
       +
           icon: Optional[HttpUrl] = None

     

       41
       41
       +
           image_url: Optional[HttpUrl] = None

     

       42
       42
       +
           description: Optional[str] = None

     

       43
       43
       +
       

     

       44
       44
       +
           def to_user_config(self, username: str, feed_url: HttpUrl) -> "UserConfig":

     

       45
       45
       +
               """Convert discovered metadata to UserConfig with fallbacks."""

     

       46
       46
       +
               from .config import UserConfig

     

       47
       47
       +
       

     

       48
       48
       +
               return UserConfig(

     

       49
       49
       +
                   username=username,

     

       50
       50
       +
                   feeds=[feed_url],

     

       51
       51
       +
                   display_name=self.author_name or self.title,

     

       52
       52
       +
                   email=self.author_email,

     

       53
       53
       +
                   homepage=self.author_uri or self.link,

     

       54
       54
       +
                   icon=self.logo or self.icon or self.image_url,

     

       55
       55
       +
               )

     

       56
       56
       +
       

     

       57
       57
       +
       

     

       58
       58
       +
       class DuplicateMap(BaseModel):

     

       59
       59
       +
           """Maps duplicate entry IDs to canonical entry IDs."""

     

       60
       60
       +
       

     

       61
       61
       +
           duplicates: dict[str, str] = {}  # duplicate_id -> canonical_id

     

       62
       62
       +
           comment: str = "Entry IDs that map to the same canonical content"

     

       63
       63
       +
       

     

       64
       64
       +
           def add_duplicate(self, duplicate_id: str, canonical_id: str) -> None:

     

       65
       65
       +
               """Add a duplicate mapping."""

     

       66
       66
       +
               self.duplicates[duplicate_id] = canonical_id

     

       67
       67
       +
       

     

       68
       68
       +
           def remove_duplicate(self, duplicate_id: str) -> bool:

     

       69
       69
       +
               """Remove a duplicate mapping. Returns True if existed."""

     

       70
       70
       +
               return self.duplicates.pop(duplicate_id, None) is not None

     

       71
       71
       +
       

     

       72
       72
       +
           def get_canonical(self, entry_id: str) -> str:

     

       73
       73
       +
               """Get canonical ID for an entry (returns original if not duplicate)."""

     

       74
       74
       +
               return self.duplicates.get(entry_id, entry_id)

     

       75
       75
       +
       

     

       76
       76
       +
           def is_duplicate(self, entry_id: str) -> bool:

     

       77
       77
       +
               """Check if entry ID is marked as duplicate."""

     

       78
       78
       +
               return entry_id in self.duplicates

     

       79
       79
       +
       

     

       80
       80
       +
           def get_duplicates_for_canonical(self, canonical_id: str) -> list[str]:

     

       81
       81
       +
               """Get all duplicate IDs that map to a canonical ID."""

     

       82
       82
       +
               return [

     

       83
       83
       +
                   duplicate_id

     

       84
       84
       +
                   for duplicate_id, canonical in self.duplicates.items()

     

       85
       85
       +
                   if canonical == canonical_id

     

       86
       86
       +
               ]

+79

src/thicket/models/user.py

···

       1
       1
       +
       """User metadata models for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       from datetime import datetime

     

       4
       4
       +
       from typing import Optional

     

       5
       5
       +
       

     

       6
       6
       +
       from pydantic import BaseModel, ConfigDict

     

       7
       7
       +
       

     

       8
       8
       +
       

     

       9
       9
       +
       class UserMetadata(BaseModel):

     

       10
       10
       +
           """Metadata about a user stored in the Git repository."""

     

       11
       11
       +
       

     

       12
       12
       +
           model_config = ConfigDict(

     

       13
       13
       +
               json_encoders={datetime: lambda v: v.isoformat()},

     

       14
       14
       +
               str_strip_whitespace=True,

     

       15
       15
       +
           )

     

       16
       16
       +
       

     

       17
       17
       +
           username: str

     

       18
       18
       +
           display_name: Optional[str] = None

     

       19
       19
       +
           email: Optional[str] = None

     

       20
       20
       +
           homepage: Optional[str] = None

     

       21
       21
       +
           icon: Optional[str] = None

     

       22
       22
       +
           feeds: list[str] = []

     

       23
       23
       +
           directory: str  # Directory name in Git store

     

       24
       24
       +
           created: datetime

     

       25
       25
       +
           last_updated: datetime

     

       26
       26
       +
           entry_count: int = 0

     

       27
       27
       +
       

     

       28
       28
       +
           def update_timestamp(self) -> None:

     

       29
       29
       +
               """Update the last_updated timestamp to now."""

     

       30
       30
       +
               self.last_updated = datetime.now()

     

       31
       31
       +
       

     

       32
       32
       +
           def increment_entry_count(self, count: int = 1) -> None:

     

       33
       33
       +
               """Increment the entry count by the given amount."""

     

       34
       34
       +
               self.entry_count += count

     

       35
       35
       +
               self.update_timestamp()

     

       36
       36
       +
       

     

       37
       37
       +
       

     

       38
       38
       +
       class GitStoreIndex(BaseModel):

     

       39
       39
       +
           """Index of all users and their directories in the Git store."""

     

       40
       40
       +
       

     

       41
       41
       +
           model_config = ConfigDict(

     

       42
       42
       +
               json_encoders={datetime: lambda v: v.isoformat()}

     

       43
       43
       +
           )

     

       44
       44
       +
       

     

       45
       45
       +
           users: dict[str, UserMetadata] = {}  # username -> UserMetadata

     

       46
       46
       +
           created: datetime

     

       47
       47
       +
           last_updated: datetime

     

       48
       48
       +
           total_entries: int = 0

     

       49
       49
       +
       

     

       50
       50
       +
           def add_user(self, user_metadata: UserMetadata) -> None:

     

       51
       51
       +
               """Add or update a user in the index."""

     

       52
       52
       +
               self.users[user_metadata.username] = user_metadata

     

       53
       53
       +
               self.last_updated = datetime.now()

     

       54
       54
       +
       

     

       55
       55
       +
           def remove_user(self, username: str) -> bool:

     

       56
       56
       +
               """Remove a user from the index. Returns True if user existed."""

     

       57
       57
       +
               if username in self.users:

     

       58
       58
       +
                   del self.users[username]

     

       59
       59
       +
                   self.last_updated = datetime.now()

     

       60
       60
       +
                   return True

     

       61
       61
       +
               return False

     

       62
       62
       +
       

     

       63
       63
       +
           def get_user(self, username: str) -> Optional[UserMetadata]:

     

       64
       64
       +
               """Get user metadata by username."""

     

       65
       65
       +
               return self.users.get(username)

     

       66
       66
       +
       

     

       67
       67
       +
           def update_entry_count(self, username: str, count: int) -> None:

     

       68
       68
       +
               """Update entry count for a user and total."""

     

       69
       69
       +
               user = self.get_user(username)

     

       70
       70
       +
               if user:

     

       71
       71
       +
                   old_count = user.entry_count

     

       72
       72
       +
                   user.increment_entry_count(count)

     

       73
       73
       +
                   self.total_entries += count

     

       74
       74
       +
                   self.last_updated = datetime.now()

     

       75
       75
       +
       

     

       76
       76
       +
           def recalculate_totals(self) -> None:

     

       77
       77
       +
               """Recalculate total entries from all users."""

     

       78
       78
       +
               self.total_entries = sum(user.entry_count for user in self.users.values())

     

       79
       79
       +
               self.last_updated = datetime.now()

src/thicket/utils/__init__.py

···

       1
       1
       +
       """Utility modules for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       # This module will contain shared utilities

     

       4
       4
       +
       # For now, it's empty but can be expanded with common functions

tests/__init__.py

This is a binary file and will not be displayed.

+84

tests/conftest.py

···

       1
       1
       +
       """Test configuration and fixtures for thicket."""

     

       2
       2
       +
       

     

       3
       3
       +
       import tempfile

     

       4
       4
       +
       from pathlib import Path

     

       5
       5
       +
       

     

       6
       6
       +
       import pytest

     

       7
       7
       +
       

     

       8
       8
       +
       from thicket.models import ThicketConfig, UserConfig

     

       9
       9
       +
       

     

       10
       10
       +
       

     

       11
       11
       +
       @pytest.fixture

     

       12
       12
       +
       def temp_dir():

     

       13
       13
       +
           """Create a temporary directory for tests."""

     

       14
       14
       +
           with tempfile.TemporaryDirectory() as tmp_dir:

     

       15
       15
       +
               yield Path(tmp_dir)

     

       16
       16
       +
       

     

       17
       17
       +
       

     

       18
       18
       +
       @pytest.fixture

     

       19
       19
       +
       def sample_config(temp_dir):

     

       20
       20
       +
           """Create a sample configuration for testing."""

     

       21
       21
       +
           git_store = temp_dir / "git_store"

     

       22
       22
       +
           cache_dir = temp_dir / "cache"

     

       23
       23
       +
           

     

       24
       24
       +
           return ThicketConfig(

     

       25
       25
       +
               git_store=git_store,

     

       26
       26
       +
               cache_dir=cache_dir,

     

       27
       27
       +
               users=[

     

       28
       28
       +
                   UserConfig(

     

       29
       29
       +
                       username="testuser",

     

       30
       30
       +
                       feeds=["https://example.com/feed.xml"],

     

       31
       31
       +
                       email="test@example.com",

     

       32
       32
       +
                       display_name="Test User",

     

       33
       33
       +
                   )

     

       34
       34
       +
               ],

     

       35
       35
       +
           )

     

       36
       36
       +
       

     

       37
       37
       +
       

     

       38
       38
       +
       @pytest.fixture

     

       39
       39
       +
       def sample_atom_feed():

     

       40
       40
       +
           """Sample Atom feed XML for testing."""

     

       41
       41
       +
           return """<?xml version="1.0" encoding="utf-8"?>

     

       42
       42
       +
       <feed xmlns="http://www.w3.org/2005/Atom">

     

       43
       43
       +
         <title>Test Feed</title>

     

       44
       44
       +
         <link href="https://example.com/"/>

     

       45
       45
       +
         <updated>2025-01-01T00:00:00Z</updated>

     

       46
       46
       +
         <author>

     

       47
       47
       +
           <name>Test Author</name>

     

       48
       48
       +
           <email>author@example.com</email>

     

       49
       49
       +
         </author>

     

       50
       50
       +
         <id>https://example.com/</id>

     

       51
       51
       +
       

     

       52
       52
       +
         <entry>

     

       53
       53
       +
           <title>Test Entry</title>

     

       54
       54
       +
           <link href="https://example.com/entry/1"/>

     

       55
       55
       +
           <id>https://example.com/entry/1</id>

     

       56
       56
       +
           <updated>2025-01-01T00:00:00Z</updated>

     

       57
       57
       +
           <summary>This is a test entry.</summary>

     

       58
       58
       +
           <content type="html">

     

       59
       59
       +
             <![CDATA[<p>This is the content of the test entry.</p>]]>

     

       60
       60
       +
           </content>

     

       61
       61
       +
         </entry>

     

       62
       62
       +
       </feed>"""

     

       63
       63
       +
       

     

       64
       64
       +
       

     

       65
       65
       +
       @pytest.fixture

     

       66
       66
       +
       def sample_rss_feed():

     

       67
       67
       +
           """Sample RSS feed XML for testing."""

     

       68
       68
       +
           return """<?xml version="1.0" encoding="UTF-8"?>

     

       69
       69
       +
       <rss version="2.0">

     

       70
       70
       +
         <channel>

     

       71
       71
       +
           <title>Test RSS Feed</title>

     

       72
       72
       +
           <link>https://example.com/</link>

     

       73
       73
       +
           <description>Test RSS feed for testing</description>

     

       74
       74
       +
           <managingEditor>editor@example.com</managingEditor>

     

       75
       75
       +
           

     

       76
       76
       +
           <item>

     

       77
       77
       +
             <title>Test RSS Entry</title>

     

       78
       78
       +
             <link>https://example.com/rss/entry/1</link>

     

       79
       79
       +
             <description>This is a test RSS entry.</description>

     

       80
       80
       +
             <pubDate>Mon, 01 Jan 2025 00:00:00 GMT</pubDate>

     

       81
       81
       +
             <guid>https://example.com/rss/entry/1</guid>

     

       82
       82
       +
           </item>

     

       83
       83
       +
         </channel>

     

       84
       84
       +
       </rss>"""

+132

tests/test_feed_parser.py

···

       1
       1
       +
       """Tests for feed parser functionality."""

     

       2
       2
       +
       

     

       3
       3
       +
       import pytest

     

       4
       4
       +
       from pydantic import HttpUrl

     

       5
       5
       +
       

     

       6
       6
       +
       from thicket.core.feed_parser import FeedParser

     

       7
       7
       +
       from thicket.models import AtomEntry, FeedMetadata

     

       8
       8
       +
       

     

       9
       9
       +
       

     

       10
       10
       +
       class TestFeedParser:

     

       11
       11
       +
           """Test the FeedParser class."""

     

       12
       12
       +
       

     

       13
       13
       +
           def test_init(self):

     

       14
       14
       +
               """Test parser initialization."""

     

       15
       15
       +
               parser = FeedParser()

     

       16
       16
       +
               assert parser.user_agent == "thicket/0.1.0"

     

       17
       17
       +
               assert "a" in parser.allowed_tags

     

       18
       18
       +
               assert "href" in parser.allowed_attributes["a"]

     

       19
       19
       +
       

     

       20
       20
       +
           def test_parse_atom_feed(self, sample_atom_feed):

     

       21
       21
       +
               """Test parsing an Atom feed."""

     

       22
       22
       +
               parser = FeedParser()

     

       23
       23
       +
               metadata, entries = parser.parse_feed(sample_atom_feed)

     

       24
       24
       +
               

     

       25
       25
       +
               # Check metadata

     

       26
       26
       +
               assert isinstance(metadata, FeedMetadata)

     

       27
       27
       +
               assert metadata.title == "Test Feed"

     

       28
       28
       +
               assert metadata.author_name == "Test Author"

     

       29
       29
       +
               assert metadata.author_email == "author@example.com"

     

       30
       30
       +
               assert metadata.link == HttpUrl("https://example.com/")

     

       31
       31
       +
               

     

       32
       32
       +
               # Check entries

     

       33
       33
       +
               assert len(entries) == 1

     

       34
       34
       +
               entry = entries[0]

     

       35
       35
       +
               assert isinstance(entry, AtomEntry)

     

       36
       36
       +
               assert entry.title == "Test Entry"

     

       37
       37
       +
               assert entry.id == "https://example.com/entry/1"

     

       38
       38
       +
               assert entry.link == HttpUrl("https://example.com/entry/1")

     

       39
       39
       +
               assert entry.summary == "This is a test entry."

     

       40
       40
       +
               assert "<p>This is the content of the test entry.</p>" in entry.content

     

       41
       41
       +
       

     

       42
       42
       +
           def test_parse_rss_feed(self, sample_rss_feed):

     

       43
       43
       +
               """Test parsing an RSS feed."""

     

       44
       44
       +
               parser = FeedParser()

     

       45
       45
       +
               metadata, entries = parser.parse_feed(sample_rss_feed)

     

       46
       46
       +
               

     

       47
       47
       +
               # Check metadata

     

       48
       48
       +
               assert isinstance(metadata, FeedMetadata)

     

       49
       49
       +
               assert metadata.title == "Test RSS Feed"

     

       50
       50
       +
               assert metadata.link == HttpUrl("https://example.com/")

     

       51
       51
       +
               assert metadata.author_email == "editor@example.com"

     

       52
       52
       +
               

     

       53
       53
       +
               # Check entries

     

       54
       54
       +
               assert len(entries) == 1

     

       55
       55
       +
               entry = entries[0]

     

       56
       56
       +
               assert isinstance(entry, AtomEntry)

     

       57
       57
       +
               assert entry.title == "Test RSS Entry"

     

       58
       58
       +
               assert entry.id == "https://example.com/rss/entry/1"

     

       59
       59
       +
               assert entry.summary == "This is a test RSS entry."

     

       60
       60
       +
       

     

       61
       61
       +
           def test_sanitize_entry_id(self):

     

       62
       62
       +
               """Test entry ID sanitization."""

     

       63
       63
       +
               parser = FeedParser()

     

       64
       64
       +
               

     

       65
       65
       +
               # Test URL ID

     

       66
       66
       +
               url_id = "https://example.com/posts/2025/01/test-post"

     

       67
       67
       +
               sanitized = parser.sanitize_entry_id(url_id)

     

       68
       68
       +
               assert sanitized == "posts_2025_01_test-post"

     

       69
       69
       +
               

     

       70
       70
       +
               # Test problematic characters

     

       71
       71
       +
               bad_id = "test/with\\bad:chars|and<more>"

     

       72
       72
       +
               sanitized = parser.sanitize_entry_id(bad_id)

     

       73
       73
       +
               assert sanitized == "test_with_bad_chars_and_more_"

     

       74
       74
       +
               

     

       75
       75
       +
               # Test empty ID

     

       76
       76
       +
               empty_id = ""

     

       77
       77
       +
               sanitized = parser.sanitize_entry_id(empty_id)

     

       78
       78
       +
               assert sanitized == "entry"

     

       79
       79
       +
               

     

       80
       80
       +
               # Test very long ID

     

       81
       81
       +
               long_id = "a" * 300

     

       82
       82
       +
               sanitized = parser.sanitize_entry_id(long_id)

     

       83
       83
       +
               assert len(sanitized) == 200

     

       84
       84
       +
       

     

       85
       85
       +
           def test_sanitize_html(self):

     

       86
       86
       +
               """Test HTML sanitization."""

     

       87
       87
       +
               parser = FeedParser()

     

       88
       88
       +
               

     

       89
       89
       +
               # Test allowed tags

     

       90
       90
       +
               safe_html = "<p>This is <strong>safe</strong> HTML</p>"

     

       91
       91
       +
               sanitized = parser._sanitize_html(safe_html)

     

       92
       92
       +
               assert sanitized == safe_html

     

       93
       93
       +
               

     

       94
       94
       +
               # Test dangerous tags

     

       95
       95
       +
               dangerous_html = "<script>alert('xss')</script><p>Safe content</p>"

     

       96
       96
       +
               sanitized = parser._sanitize_html(dangerous_html)

     

       97
       97
       +
               assert "<script>" not in sanitized

     

       98
       98
       +
               assert "<p>Safe content</p>" in sanitized

     

       99
       99
       +
               

     

       100
       100
       +
               # Test attributes

     

       101
       101
       +
               html_with_attrs = '<a href="https://example.com" onclick="alert()">Link</a>'

     

       102
       102
       +
               sanitized = parser._sanitize_html(html_with_attrs)

     

       103
       103
       +
               assert 'href="https://example.com"' in sanitized

     

       104
       104
       +
               assert 'onclick' not in sanitized

     

       105
       105
       +
       

     

       106
       106
       +
           def test_extract_feed_metadata(self):

     

       107
       107
       +
               """Test feed metadata extraction."""

     

       108
       108
       +
               parser = FeedParser()

     

       109
       109
       +
               

     

       110
       110
       +
               # Test with feedparser parsed data

     

       111
       111
       +
               import feedparser

     

       112
       112
       +
               parsed = feedparser.parse("""<?xml version="1.0" encoding="utf-8"?>

     

       113
       113
       +
       <feed xmlns="http://www.w3.org/2005/Atom">

     

       114
       114
       +
         <title>Test Feed</title>

     

       115
       115
       +
         <link href="https://example.com/"/>

     

       116
       116
       +
         <author>

     

       117
       117
       +
           <name>Test Author</name>

     

       118
       118
       +
           <email>author@example.com</email>

     

       119
       119
       +
           <uri>https://example.com/about</uri>

     

       120
       120
       +
         </author>

     

       121
       121
       +
         <logo>https://example.com/logo.png</logo>

     

       122
       122
       +
         <icon>https://example.com/icon.png</icon>

     

       123
       123
       +
       </feed>""")

     

       124
       124
       +
               

     

       125
       125
       +
               metadata = parser._extract_feed_metadata(parsed.feed)

     

       126
       126
       +
               assert metadata.title == "Test Feed"

     

       127
       127
       +
               assert metadata.author_name == "Test Author"

     

       128
       128
       +
               assert metadata.author_email == "author@example.com"

     

       129
       129
       +
               assert metadata.author_uri == HttpUrl("https://example.com/about")

     

       130
       130
       +
               assert metadata.link == HttpUrl("https://example.com/")

     

       131
       131
       +
               assert metadata.logo == HttpUrl("https://example.com/logo.png")

     

       132
       132
       +
               assert metadata.icon == HttpUrl("https://example.com/icon.png")

+277

tests/test_git_store.py

···

       1
       1
       +
       """Tests for Git store functionality."""

     

       2
       2
       +
       

     

       3
       3
       +
       import json

     

       4
       4
       +
       from datetime import datetime

     

       5
       5
       +
       

     

       6
       6
       +
       import pytest

     

       7
       7
       +
       from pydantic import HttpUrl

     

       8
       8
       +
       

     

       9
       9
       +
       from thicket.core.git_store import GitStore

     

       10
       10
       +
       from thicket.models import AtomEntry, DuplicateMap, UserMetadata

     

       11
       11
       +
       

     

       12
       12
       +
       

     

       13
       13
       +
       class TestGitStore:

     

       14
       14
       +
           """Test the GitStore class."""

     

       15
       15
       +
       

     

       16
       16
       +
           def test_init_new_repo(self, temp_dir):

     

       17
       17
       +
               """Test initializing a new Git repository."""

     

       18
       18
       +
               repo_path = temp_dir / "test_repo"

     

       19
       19
       +
               store = GitStore(repo_path)

     

       20
       20
       +
               

     

       21
       21
       +
               assert store.repo_path == repo_path

     

       22
       22
       +
               assert store.repo is not None

     

       23
       23
       +
               assert repo_path.exists()

     

       24
       24
       +
               assert (repo_path / ".git").exists()

     

       25
       25
       +
               assert (repo_path / "index.json").exists()

     

       26
       26
       +
               assert (repo_path / "duplicates.json").exists()

     

       27
       27
       +
       

     

       28
       28
       +
           def test_init_existing_repo(self, temp_dir):

     

       29
       29
       +
               """Test initializing with existing repository."""

     

       30
       30
       +
               repo_path = temp_dir / "test_repo"

     

       31
       31
       +
               

     

       32
       32
       +
               # Create first store

     

       33
       33
       +
               store1 = GitStore(repo_path)

     

       34
       34
       +
               store1.add_user("testuser", display_name="Test User")

     

       35
       35
       +
               

     

       36
       36
       +
               # Create second store pointing to same repo

     

       37
       37
       +
               store2 = GitStore(repo_path)

     

       38
       38
       +
               user = store2.get_user("testuser")

     

       39
       39
       +
               

     

       40
       40
       +
               assert user is not None

     

       41
       41
       +
               assert user.username == "testuser"

     

       42
       42
       +
               assert user.display_name == "Test User"

     

       43
       43
       +
       

     

       44
       44
       +
           def test_add_user(self, temp_dir):

     

       45
       45
       +
               """Test adding a user to the Git store."""

     

       46
       46
       +
               store = GitStore(temp_dir / "test_repo")

     

       47
       47
       +
               

     

       48
       48
       +
               user = store.add_user(

     

       49
       49
       +
                   username="testuser",

     

       50
       50
       +
                   display_name="Test User",

     

       51
       51
       +
                   email="test@example.com",

     

       52
       52
       +
                   homepage="https://example.com",

     

       53
       53
       +
                   icon="https://example.com/icon.png",

     

       54
       54
       +
                   feeds=["https://example.com/feed.xml"],

     

       55
       55
       +
               )

     

       56
       56
       +
               

     

       57
       57
       +
               assert isinstance(user, UserMetadata)

     

       58
       58
       +
               assert user.username == "testuser"

     

       59
       59
       +
               assert user.display_name == "Test User"

     

       60
       60
       +
               assert user.email == "test@example.com"

     

       61
       61
       +
               assert user.homepage == "https://example.com"

     

       62
       62
       +
               assert user.icon == "https://example.com/icon.png"

     

       63
       63
       +
               assert user.feeds == ["https://example.com/feed.xml"]

     

       64
       64
       +
               assert user.directory == "testuser"

     

       65
       65
       +
               

     

       66
       66
       +
               # Check that user directory was created

     

       67
       67
       +
               user_dir = store.repo_path / "testuser"

     

       68
       68
       +
               assert user_dir.exists()

     

       69
       69
       +
               assert (user_dir / "metadata.json").exists()

     

       70
       70
       +
               

     

       71
       71
       +
               # Check metadata file content

     

       72
       72
       +
               with open(user_dir / "metadata.json") as f:

     

       73
       73
       +
                   metadata = json.load(f)

     

       74
       74
       +
               assert metadata["username"] == "testuser"

     

       75
       75
       +
               assert metadata["display_name"] == "Test User"

     

       76
       76
       +
       

     

       77
       77
       +
           def test_get_user(self, temp_dir):

     

       78
       78
       +
               """Test getting user metadata."""

     

       79
       79
       +
               store = GitStore(temp_dir / "test_repo")

     

       80
       80
       +
               

     

       81
       81
       +
               # Add user

     

       82
       82
       +
               store.add_user("testuser", display_name="Test User")

     

       83
       83
       +
               

     

       84
       84
       +
               # Get user

     

       85
       85
       +
               user = store.get_user("testuser")

     

       86
       86
       +
               assert user is not None

     

       87
       87
       +
               assert user.username == "testuser"

     

       88
       88
       +
               assert user.display_name == "Test User"

     

       89
       89
       +
               

     

       90
       90
       +
               # Try to get non-existent user

     

       91
       91
       +
               non_user = store.get_user("nonexistent")

     

       92
       92
       +
               assert non_user is None

     

       93
       93
       +
       

     

       94
       94
       +
           def test_store_entry(self, temp_dir):

     

       95
       95
       +
               """Test storing an entry."""

     

       96
       96
       +
               store = GitStore(temp_dir / "test_repo")

     

       97
       97
       +
               

     

       98
       98
       +
               # Add user first

     

       99
       99
       +
               store.add_user("testuser")

     

       100
       100
       +
               

     

       101
       101
       +
               # Create test entry

     

       102
       102
       +
               entry = AtomEntry(

     

       103
       103
       +
                   id="https://example.com/entry/1",

     

       104
       104
       +
                   title="Test Entry",

     

       105
       105
       +
                   link=HttpUrl("https://example.com/entry/1"),

     

       106
       106
       +
                   updated=datetime.now(),

     

       107
       107
       +
                   summary="Test entry summary",

     

       108
       108
       +
                   content="<p>Test content</p>",

     

       109
       109
       +
               )

     

       110
       110
       +
               

     

       111
       111
       +
               # Store entry

     

       112
       112
       +
               result = store.store_entry("testuser", entry)

     

       113
       113
       +
               assert result is True

     

       114
       114
       +
               

     

       115
       115
       +
               # Check that entry file was created

     

       116
       116
       +
               user_dir = store.repo_path / "testuser"

     

       117
       117
       +
               entry_files = list(user_dir.glob("*.json"))

     

       118
       118
       +
               entry_files = [f for f in entry_files if f.name != "metadata.json"]

     

       119
       119
       +
               assert len(entry_files) == 1

     

       120
       120
       +
               

     

       121
       121
       +
               # Check entry content

     

       122
       122
       +
               with open(entry_files[0]) as f:

     

       123
       123
       +
                   stored_entry = json.load(f)

     

       124
       124
       +
               assert stored_entry["title"] == "Test Entry"

     

       125
       125
       +
               assert stored_entry["id"] == "https://example.com/entry/1"

     

       126
       126
       +
       

     

       127
       127
       +
           def test_get_entry(self, temp_dir):

     

       128
       128
       +
               """Test retrieving an entry."""

     

       129
       129
       +
               store = GitStore(temp_dir / "test_repo")

     

       130
       130
       +
               

     

       131
       131
       +
               # Add user and entry

     

       132
       132
       +
               store.add_user("testuser")

     

       133
       133
       +
               entry = AtomEntry(

     

       134
       134
       +
                   id="https://example.com/entry/1",

     

       135
       135
       +
                   title="Test Entry",

     

       136
       136
       +
                   link=HttpUrl("https://example.com/entry/1"),

     

       137
       137
       +
                   updated=datetime.now(),

     

       138
       138
       +
               )

     

       139
       139
       +
               store.store_entry("testuser", entry)

     

       140
       140
       +
               

     

       141
       141
       +
               # Get entry

     

       142
       142
       +
               retrieved = store.get_entry("testuser", "https://example.com/entry/1")

     

       143
       143
       +
               assert retrieved is not None

     

       144
       144
       +
               assert retrieved.title == "Test Entry"

     

       145
       145
       +
               assert retrieved.id == "https://example.com/entry/1"

     

       146
       146
       +
               

     

       147
       147
       +
               # Try to get non-existent entry

     

       148
       148
       +
               non_entry = store.get_entry("testuser", "https://example.com/nonexistent")

     

       149
       149
       +
               assert non_entry is None

     

       150
       150
       +
       

     

       151
       151
       +
           def test_list_entries(self, temp_dir):

     

       152
       152
       +
               """Test listing entries for a user."""

     

       153
       153
       +
               store = GitStore(temp_dir / "test_repo")

     

       154
       154
       +
               

     

       155
       155
       +
               # Add user

     

       156
       156
       +
               store.add_user("testuser")

     

       157
       157
       +
               

     

       158
       158
       +
               # Add multiple entries

     

       159
       159
       +
               for i in range(3):

     

       160
       160
       +
                   entry = AtomEntry(

     

       161
       161
       +
                       id=f"https://example.com/entry/{i}",

     

       162
       162
       +
                       title=f"Test Entry {i}",

     

       163
       163
       +
                       link=HttpUrl(f"https://example.com/entry/{i}"),

     

       164
       164
       +
                       updated=datetime.now(),

     

       165
       165
       +
                   )

     

       166
       166
       +
                   store.store_entry("testuser", entry)

     

       167
       167
       +
               

     

       168
       168
       +
               # List all entries

     

       169
       169
       +
               entries = store.list_entries("testuser")

     

       170
       170
       +
               assert len(entries) == 3

     

       171
       171
       +
               

     

       172
       172
       +
               # List with limit

     

       173
       173
       +
               limited = store.list_entries("testuser", limit=2)

     

       174
       174
       +
               assert len(limited) == 2

     

       175
       175
       +
               

     

       176
       176
       +
               # List for non-existent user

     

       177
       177
       +
               none_entries = store.list_entries("nonexistent")

     

       178
       178
       +
               assert len(none_entries) == 0

     

       179
       179
       +
       

     

       180
       180
       +
           def test_duplicates(self, temp_dir):

     

       181
       181
       +
               """Test duplicate management."""

     

       182
       182
       +
               store = GitStore(temp_dir / "test_repo")

     

       183
       183
       +
               

     

       184
       184
       +
               # Get initial duplicates (should be empty)

     

       185
       185
       +
               duplicates = store.get_duplicates()

     

       186
       186
       +
               assert isinstance(duplicates, DuplicateMap)

     

       187
       187
       +
               assert len(duplicates.duplicates) == 0

     

       188
       188
       +
               

     

       189
       189
       +
               # Add duplicate

     

       190
       190
       +
               store.add_duplicate("https://example.com/dup", "https://example.com/canonical")

     

       191
       191
       +
               

     

       192
       192
       +
               # Check duplicate was added

     

       193
       193
       +
               duplicates = store.get_duplicates()

     

       194
       194
       +
               assert len(duplicates.duplicates) == 1

     

       195
       195
       +
               assert duplicates.is_duplicate("https://example.com/dup")

     

       196
       196
       +
               assert duplicates.get_canonical("https://example.com/dup") == "https://example.com/canonical"

     

       197
       197
       +
               

     

       198
       198
       +
               # Remove duplicate

     

       199
       199
       +
               result = store.remove_duplicate("https://example.com/dup")

     

       200
       200
       +
               assert result is True

     

       201
       201
       +
               

     

       202
       202
       +
               # Check duplicate was removed

     

       203
       203
       +
               duplicates = store.get_duplicates()

     

       204
       204
       +
               assert len(duplicates.duplicates) == 0

     

       205
       205
       +
               assert not duplicates.is_duplicate("https://example.com/dup")

     

       206
       206
       +
       

     

       207
       207
       +
           def test_search_entries(self, temp_dir):

     

       208
       208
       +
               """Test searching entries."""

     

       209
       209
       +
               store = GitStore(temp_dir / "test_repo")

     

       210
       210
       +
               

     

       211
       211
       +
               # Add user

     

       212
       212
       +
               store.add_user("testuser")

     

       213
       213
       +
               

     

       214
       214
       +
               # Add entries with different content

     

       215
       215
       +
               entries_data = [

     

       216
       216
       +
                   ("Test Python Programming", "Learning Python basics"),

     

       217
       217
       +
                   ("JavaScript Tutorial", "Advanced JavaScript concepts"),

     

       218
       218
       +
                   ("Python Web Development", "Building web apps with Python"),

     

       219
       219
       +
               ]

     

       220
       220
       +
               

     

       221
       221
       +
               for title, summary in entries_data:

     

       222
       222
       +
                   entry = AtomEntry(

     

       223
       223
       +
                       id=f"https://example.com/entry/{title.lower().replace(' ', '-')}",

     

       224
       224
       +
                       title=title,

     

       225
       225
       +
                       link=HttpUrl(f"https://example.com/entry/{title.lower().replace(' ', '-')}"),

     

       226
       226
       +
                       updated=datetime.now(),

     

       227
       227
       +
                       summary=summary,

     

       228
       228
       +
                   )

     

       229
       229
       +
                   store.store_entry("testuser", entry)

     

       230
       230
       +
               

     

       231
       231
       +
               # Search for Python entries

     

       232
       232
       +
               results = store.search_entries("Python")

     

       233
       233
       +
               assert len(results) == 2

     

       234
       234
       +
               

     

       235
       235
       +
               # Search for specific user

     

       236
       236
       +
               results = store.search_entries("Python", username="testuser")

     

       237
       237
       +
               assert len(results) == 2

     

       238
       238
       +
               

     

       239
       239
       +
               # Search with limit

     

       240
       240
       +
               results = store.search_entries("Python", limit=1)

     

       241
       241
       +
               assert len(results) == 1

     

       242
       242
       +
               

     

       243
       243
       +
               # Search for non-existent term

     

       244
       244
       +
               results = store.search_entries("NonExistent")

     

       245
       245
       +
               assert len(results) == 0

     

       246
       246
       +
       

     

       247
       247
       +
           def test_get_stats(self, temp_dir):

     

       248
       248
       +
               """Test getting repository statistics."""

     

       249
       249
       +
               store = GitStore(temp_dir / "test_repo")

     

       250
       250
       +
               

     

       251
       251
       +
               # Get initial stats

     

       252
       252
       +
               stats = store.get_stats()

     

       253
       253
       +
               assert stats["total_users"] == 0

     

       254
       254
       +
               assert stats["total_entries"] == 0

     

       255
       255
       +
               assert stats["total_duplicates"] == 0

     

       256
       256
       +
               

     

       257
       257
       +
               # Add user and entries

     

       258
       258
       +
               store.add_user("testuser")

     

       259
       259
       +
               for i in range(3):

     

       260
       260
       +
                   entry = AtomEntry(

     

       261
       261
       +
                       id=f"https://example.com/entry/{i}",

     

       262
       262
       +
                       title=f"Test Entry {i}",

     

       263
       263
       +
                       link=HttpUrl(f"https://example.com/entry/{i}"),

     

       264
       264
       +
                       updated=datetime.now(),

     

       265
       265
       +
                   )

     

       266
       266
       +
                   store.store_entry("testuser", entry)

     

       267
       267
       +
               

     

       268
       268
       +
               # Add duplicate

     

       269
       269
       +
               store.add_duplicate("https://example.com/dup", "https://example.com/canonical")

     

       270
       270
       +
               

     

       271
       271
       +
               # Get updated stats

     

       272
       272
       +
               stats = store.get_stats()

     

       273
       273
       +
               assert stats["total_users"] == 1

     

       274
       274
       +
               assert stats["total_entries"] == 3

     

       275
       275
       +
               assert stats["total_duplicates"] == 1

     

       276
       276
       +
               assert "last_updated" in stats

     

       277
       277
       +
               assert "repository_size" in stats

+353

tests/test_models.py

···

       1
       1
       +
       """Tests for pydantic models."""

     

       2
       2
       +
       

     

       3
       3
       +
       from datetime import datetime

     

       4
       4
       +
       from pathlib import Path

     

       5
       5
       +
       

     

       6
       6
       +
       import pytest

     

       7
       7
       +
       from pydantic import HttpUrl, ValidationError

     

       8
       8
       +
       

     

       9
       9
       +
       from thicket.models import (

     

       10
       10
       +
           AtomEntry,

     

       11
       11
       +
           DuplicateMap,

     

       12
       12
       +
           FeedMetadata,

     

       13
       13
       +
           ThicketConfig,

     

       14
       14
       +
           UserConfig,

     

       15
       15
       +
           UserMetadata,

     

       16
       16
       +
       )

     

       17
       17
       +
       

     

       18
       18
       +
       

     

       19
       19
       +
       class TestUserConfig:

     

       20
       20
       +
           """Test UserConfig model."""

     

       21
       21
       +
       

     

       22
       22
       +
           def test_valid_user_config(self):

     

       23
       23
       +
               """Test creating valid user config."""

     

       24
       24
       +
               config = UserConfig(

     

       25
       25
       +
                   username="testuser",

     

       26
       26
       +
                   feeds=["https://example.com/feed.xml"],

     

       27
       27
       +
                   email="test@example.com",

     

       28
       28
       +
                   homepage="https://example.com",

     

       29
       29
       +
                   display_name="Test User",

     

       30
       30
       +
               )

     

       31
       31
       +
               

     

       32
       32
       +
               assert config.username == "testuser"

     

       33
       33
       +
               assert len(config.feeds) == 1

     

       34
       34
       +
               assert config.feeds[0] == HttpUrl("https://example.com/feed.xml")

     

       35
       35
       +
               assert config.email == "test@example.com"

     

       36
       36
       +
               assert config.display_name == "Test User"

     

       37
       37
       +
       

     

       38
       38
       +
           def test_invalid_email(self):

     

       39
       39
       +
               """Test validation of invalid email."""

     

       40
       40
       +
               with pytest.raises(ValidationError):

     

       41
       41
       +
                   UserConfig(

     

       42
       42
       +
                       username="testuser",

     

       43
       43
       +
                       feeds=["https://example.com/feed.xml"],

     

       44
       44
       +
                       email="invalid-email",

     

       45
       45
       +
                   )

     

       46
       46
       +
       

     

       47
       47
       +
           def test_invalid_feed_url(self):

     

       48
       48
       +
               """Test validation of invalid feed URL."""

     

       49
       49
       +
               with pytest.raises(ValidationError):

     

       50
       50
       +
                   UserConfig(

     

       51
       51
       +
                       username="testuser",

     

       52
       52
       +
                       feeds=["not-a-url"],

     

       53
       53
       +
                   )

     

       54
       54
       +
       

     

       55
       55
       +
           def test_optional_fields(self):

     

       56
       56
       +
               """Test optional fields with None values."""

     

       57
       57
       +
               config = UserConfig(

     

       58
       58
       +
                   username="testuser",

     

       59
       59
       +
                   feeds=["https://example.com/feed.xml"],

     

       60
       60
       +
               )

     

       61
       61
       +
               

     

       62
       62
       +
               assert config.email is None

     

       63
       63
       +
               assert config.homepage is None

     

       64
       64
       +
               assert config.icon is None

     

       65
       65
       +
               assert config.display_name is None

     

       66
       66
       +
       

     

       67
       67
       +
       

     

       68
       68
       +
       class TestThicketConfig:

     

       69
       69
       +
           """Test ThicketConfig model."""

     

       70
       70
       +
       

     

       71
       71
       +
           def test_valid_config(self, temp_dir):

     

       72
       72
       +
               """Test creating valid configuration."""

     

       73
       73
       +
               config = ThicketConfig(

     

       74
       74
       +
                   git_store=temp_dir / "git_store",

     

       75
       75
       +
                   cache_dir=temp_dir / "cache",

     

       76
       76
       +
                   users=[

     

       77
       77
       +
                       UserConfig(

     

       78
       78
       +
                           username="testuser",

     

       79
       79
       +
                           feeds=["https://example.com/feed.xml"],

     

       80
       80
       +
                       )

     

       81
       81
       +
                   ],

     

       82
       82
       +
               )

     

       83
       83
       +
               

     

       84
       84
       +
               assert config.git_store == temp_dir / "git_store"

     

       85
       85
       +
               assert config.cache_dir == temp_dir / "cache"

     

       86
       86
       +
               assert len(config.users) == 1

     

       87
       87
       +
               assert config.users[0].username == "testuser"

     

       88
       88
       +
       

     

       89
       89
       +
           def test_find_user(self, temp_dir):

     

       90
       90
       +
               """Test finding user by username."""

     

       91
       91
       +
               config = ThicketConfig(

     

       92
       92
       +
                   git_store=temp_dir / "git_store",

     

       93
       93
       +
                   cache_dir=temp_dir / "cache",

     

       94
       94
       +
                   users=[

     

       95
       95
       +
                       UserConfig(username="user1", feeds=["https://example.com/feed1.xml"]),

     

       96
       96
       +
                       UserConfig(username="user2", feeds=["https://example.com/feed2.xml"]),

     

       97
       97
       +
                   ],

     

       98
       98
       +
               )

     

       99
       99
       +
               

     

       100
       100
       +
               user = config.find_user("user1")

     

       101
       101
       +
               assert user is not None

     

       102
       102
       +
               assert user.username == "user1"

     

       103
       103
       +
               

     

       104
       104
       +
               non_user = config.find_user("nonexistent")

     

       105
       105
       +
               assert non_user is None

     

       106
       106
       +
       

     

       107
       107
       +
           def test_add_user(self, temp_dir):

     

       108
       108
       +
               """Test adding a new user."""

     

       109
       109
       +
               config = ThicketConfig(

     

       110
       110
       +
                   git_store=temp_dir / "git_store",

     

       111
       111
       +
                   cache_dir=temp_dir / "cache",

     

       112
       112
       +
                   users=[],

     

       113
       113
       +
               )

     

       114
       114
       +
               

     

       115
       115
       +
               new_user = UserConfig(

     

       116
       116
       +
                   username="newuser",

     

       117
       117
       +
                   feeds=["https://example.com/feed.xml"],

     

       118
       118
       +
               )

     

       119
       119
       +
               

     

       120
       120
       +
               config.add_user(new_user)

     

       121
       121
       +
               assert len(config.users) == 1

     

       122
       122
       +
               assert config.users[0].username == "newuser"

     

       123
       123
       +
       

     

       124
       124
       +
           def test_add_feed_to_user(self, temp_dir):

     

       125
       125
       +
               """Test adding feed to existing user."""

     

       126
       126
       +
               config = ThicketConfig(

     

       127
       127
       +
                   git_store=temp_dir / "git_store",

     

       128
       128
       +
                   cache_dir=temp_dir / "cache",

     

       129
       129
       +
                   users=[

     

       130
       130
       +
                       UserConfig(username="testuser", feeds=["https://example.com/feed1.xml"]),

     

       131
       131
       +
                   ],

     

       132
       132
       +
               )

     

       133
       133
       +
               

     

       134
       134
       +
               result = config.add_feed_to_user("testuser", HttpUrl("https://example.com/feed2.xml"))

     

       135
       135
       +
               assert result is True

     

       136
       136
       +
               

     

       137
       137
       +
               user = config.find_user("testuser")

     

       138
       138
       +
               assert len(user.feeds) == 2

     

       139
       139
       +
               assert HttpUrl("https://example.com/feed2.xml") in user.feeds

     

       140
       140
       +
               

     

       141
       141
       +
               # Test adding to non-existent user

     

       142
       142
       +
               result = config.add_feed_to_user("nonexistent", HttpUrl("https://example.com/feed.xml"))

     

       143
       143
       +
               assert result is False

     

       144
       144
       +
       

     

       145
       145
       +
       

     

       146
       146
       +
       class TestAtomEntry:

     

       147
       147
       +
           """Test AtomEntry model."""

     

       148
       148
       +
       

     

       149
       149
       +
           def test_valid_entry(self):

     

       150
       150
       +
               """Test creating valid Atom entry."""

     

       151
       151
       +
               entry = AtomEntry(

     

       152
       152
       +
                   id="https://example.com/entry/1",

     

       153
       153
       +
                   title="Test Entry",

     

       154
       154
       +
                   link=HttpUrl("https://example.com/entry/1"),

     

       155
       155
       +
                   updated=datetime.now(),

     

       156
       156
       +
                   published=datetime.now(),

     

       157
       157
       +
                   summary="Test summary",

     

       158
       158
       +
                   content="<p>Test content</p>",

     

       159
       159
       +
                   content_type="html",

     

       160
       160
       +
                   author={"name": "Test Author"},

     

       161
       161
       +
                   categories=["test", "example"],

     

       162
       162
       +
               )

     

       163
       163
       +
               

     

       164
       164
       +
               assert entry.id == "https://example.com/entry/1"

     

       165
       165
       +
               assert entry.title == "Test Entry"

     

       166
       166
       +
               assert entry.summary == "Test summary"

     

       167
       167
       +
               assert entry.content == "<p>Test content</p>"

     

       168
       168
       +
               assert entry.content_type == "html"

     

       169
       169
       +
               assert entry.author["name"] == "Test Author"

     

       170
       170
       +
               assert "test" in entry.categories

     

       171
       171
       +
       

     

       172
       172
       +
           def test_minimal_entry(self):

     

       173
       173
       +
               """Test creating minimal Atom entry."""

     

       174
       174
       +
               entry = AtomEntry(

     

       175
       175
       +
                   id="https://example.com/entry/1",

     

       176
       176
       +
                   title="Test Entry",

     

       177
       177
       +
                   link=HttpUrl("https://example.com/entry/1"),

     

       178
       178
       +
                   updated=datetime.now(),

     

       179
       179
       +
               )

     

       180
       180
       +
               

     

       181
       181
       +
               assert entry.id == "https://example.com/entry/1"

     

       182
       182
       +
               assert entry.title == "Test Entry"

     

       183
       183
       +
               assert entry.published is None

     

       184
       184
       +
               assert entry.summary is None

     

       185
       185
       +
               assert entry.content is None

     

       186
       186
       +
               assert entry.content_type == "html"  # default

     

       187
       187
       +
               assert entry.author is None

     

       188
       188
       +
               assert entry.categories == []

     

       189
       189
       +
       

     

       190
       190
       +
       

     

       191
       191
       +
       class TestDuplicateMap:

     

       192
       192
       +
           """Test DuplicateMap model."""

     

       193
       193
       +
       

     

       194
       194
       +
           def test_empty_duplicates(self):

     

       195
       195
       +
               """Test empty duplicate map."""

     

       196
       196
       +
               dup_map = DuplicateMap()

     

       197
       197
       +
               assert len(dup_map.duplicates) == 0

     

       198
       198
       +
               assert not dup_map.is_duplicate("test")

     

       199
       199
       +
               assert dup_map.get_canonical("test") == "test"

     

       200
       200
       +
       

     

       201
       201
       +
           def test_add_duplicate(self):

     

       202
       202
       +
               """Test adding duplicate mapping."""

     

       203
       203
       +
               dup_map = DuplicateMap()

     

       204
       204
       +
               dup_map.add_duplicate("dup1", "canonical1")

     

       205
       205
       +
               

     

       206
       206
       +
               assert len(dup_map.duplicates) == 1

     

       207
       207
       +
               assert dup_map.is_duplicate("dup1")

     

       208
       208
       +
               assert dup_map.get_canonical("dup1") == "canonical1"

     

       209
       209
       +
               assert dup_map.get_canonical("canonical1") == "canonical1"

     

       210
       210
       +
       

     

       211
       211
       +
           def test_remove_duplicate(self):

     

       212
       212
       +
               """Test removing duplicate mapping."""

     

       213
       213
       +
               dup_map = DuplicateMap()

     

       214
       214
       +
               dup_map.add_duplicate("dup1", "canonical1")

     

       215
       215
       +
               

     

       216
       216
       +
               result = dup_map.remove_duplicate("dup1")

     

       217
       217
       +
               assert result is True

     

       218
       218
       +
               assert len(dup_map.duplicates) == 0

     

       219
       219
       +
               assert not dup_map.is_duplicate("dup1")

     

       220
       220
       +
               

     

       221
       221
       +
               # Test removing non-existent duplicate

     

       222
       222
       +
               result = dup_map.remove_duplicate("nonexistent")

     

       223
       223
       +
               assert result is False

     

       224
       224
       +
       

     

       225
       225
       +
           def test_get_duplicates_for_canonical(self):

     

       226
       226
       +
               """Test getting all duplicates for a canonical ID."""

     

       227
       227
       +
               dup_map = DuplicateMap()

     

       228
       228
       +
               dup_map.add_duplicate("dup1", "canonical1")

     

       229
       229
       +
               dup_map.add_duplicate("dup2", "canonical1")

     

       230
       230
       +
               dup_map.add_duplicate("dup3", "canonical2")

     

       231
       231
       +
               

     

       232
       232
       +
               dups = dup_map.get_duplicates_for_canonical("canonical1")

     

       233
       233
       +
               assert len(dups) == 2

     

       234
       234
       +
               assert "dup1" in dups

     

       235
       235
       +
               assert "dup2" in dups

     

       236
       236
       +
               

     

       237
       237
       +
               dups = dup_map.get_duplicates_for_canonical("canonical2")

     

       238
       238
       +
               assert len(dups) == 1

     

       239
       239
       +
               assert "dup3" in dups

     

       240
       240
       +
               

     

       241
       241
       +
               dups = dup_map.get_duplicates_for_canonical("nonexistent")

     

       242
       242
       +
               assert len(dups) == 0

     

       243
       243
       +
       

     

       244
       244
       +
       

     

       245
       245
       +
       class TestFeedMetadata:

     

       246
       246
       +
           """Test FeedMetadata model."""

     

       247
       247
       +
       

     

       248
       248
       +
           def test_valid_metadata(self):

     

       249
       249
       +
               """Test creating valid feed metadata."""

     

       250
       250
       +
               metadata = FeedMetadata(

     

       251
       251
       +
                   title="Test Feed",

     

       252
       252
       +
                   author_name="Test Author",

     

       253
       253
       +
                   author_email="author@example.com",

     

       254
       254
       +
                   author_uri=HttpUrl("https://example.com/author"),

     

       255
       255
       +
                   link=HttpUrl("https://example.com"),

     

       256
       256
       +
                   description="Test description",

     

       257
       257
       +
               )

     

       258
       258
       +
               

     

       259
       259
       +
               assert metadata.title == "Test Feed"

     

       260
       260
       +
               assert metadata.author_name == "Test Author"

     

       261
       261
       +
               assert metadata.author_email == "author@example.com"

     

       262
       262
       +
               assert metadata.link == HttpUrl("https://example.com")

     

       263
       263
       +
       

     

       264
       264
       +
           def test_to_user_config(self):

     

       265
       265
       +
               """Test converting metadata to user config."""

     

       266
       266
       +
               metadata = FeedMetadata(

     

       267
       267
       +
                   title="Test Feed",

     

       268
       268
       +
                   author_name="Test Author",

     

       269
       269
       +
                   author_email="author@example.com",

     

       270
       270
       +
                   author_uri=HttpUrl("https://example.com/author"),

     

       271
       271
       +
                   link=HttpUrl("https://example.com"),

     

       272
       272
       +
                   logo=HttpUrl("https://example.com/logo.png"),

     

       273
       273
       +
               )

     

       274
       274
       +
               

     

       275
       275
       +
               feed_url = HttpUrl("https://example.com/feed.xml")

     

       276
       276
       +
               user_config = metadata.to_user_config("testuser", feed_url)

     

       277
       277
       +
               

     

       278
       278
       +
               assert user_config.username == "testuser"

     

       279
       279
       +
               assert user_config.feeds == [feed_url]

     

       280
       280
       +
               assert user_config.display_name == "Test Author"

     

       281
       281
       +
               assert user_config.email == "author@example.com"

     

       282
       282
       +
               assert user_config.homepage == HttpUrl("https://example.com/author")

     

       283
       283
       +
               assert user_config.icon == HttpUrl("https://example.com/logo.png")

     

       284
       284
       +
       

     

       285
       285
       +
           def test_to_user_config_fallbacks(self):

     

       286
       286
       +
               """Test fallback logic in to_user_config."""

     

       287
       287
       +
               metadata = FeedMetadata(

     

       288
       288
       +
                   title="Test Feed",

     

       289
       289
       +
                   link=HttpUrl("https://example.com"),

     

       290
       290
       +
                   icon=HttpUrl("https://example.com/icon.png"),

     

       291
       291
       +
               )

     

       292
       292
       +
               

     

       293
       293
       +
               feed_url = HttpUrl("https://example.com/feed.xml")

     

       294
       294
       +
               user_config = metadata.to_user_config("testuser", feed_url)

     

       295
       295
       +
               

     

       296
       296
       +
               assert user_config.display_name == "Test Feed"  # Falls back to title

     

       297
       297
       +
               assert user_config.homepage == HttpUrl("https://example.com")  # Falls back to link

     

       298
       298
       +
               assert user_config.icon == HttpUrl("https://example.com/icon.png")

     

       299
       299
       +
               assert user_config.email is None

     

       300
       300
       +
       

     

       301
       301
       +
       

     

       302
       302
       +
       class TestUserMetadata:

     

       303
       303
       +
           """Test UserMetadata model."""

     

       304
       304
       +
       

     

       305
       305
       +
           def test_valid_metadata(self):

     

       306
       306
       +
               """Test creating valid user metadata."""

     

       307
       307
       +
               now = datetime.now()

     

       308
       308
       +
               metadata = UserMetadata(

     

       309
       309
       +
                   username="testuser",

     

       310
       310
       +
                   directory="testuser",

     

       311
       311
       +
                   created=now,

     

       312
       312
       +
                   last_updated=now,

     

       313
       313
       +
                   feeds=["https://example.com/feed.xml"],

     

       314
       314
       +
                   entry_count=5,

     

       315
       315
       +
               )

     

       316
       316
       +
               

     

       317
       317
       +
               assert metadata.username == "testuser"

     

       318
       318
       +
               assert metadata.directory == "testuser"

     

       319
       319
       +
               assert metadata.entry_count == 5

     

       320
       320
       +
               assert len(metadata.feeds) == 1

     

       321
       321
       +
       

     

       322
       322
       +
           def test_update_timestamp(self):

     

       323
       323
       +
               """Test updating timestamp."""

     

       324
       324
       +
               now = datetime.now()

     

       325
       325
       +
               metadata = UserMetadata(

     

       326
       326
       +
                   username="testuser",

     

       327
       327
       +
                   directory="testuser",

     

       328
       328
       +
                   created=now,

     

       329
       329
       +
                   last_updated=now,

     

       330
       330
       +
               )

     

       331
       331
       +
               

     

       332
       332
       +
               original_time = metadata.last_updated

     

       333
       333
       +
               metadata.update_timestamp()

     

       334
       334
       +
               

     

       335
       335
       +
               assert metadata.last_updated > original_time

     

       336
       336
       +
       

     

       337
       337
       +
           def test_increment_entry_count(self):

     

       338
       338
       +
               """Test incrementing entry count."""

     

       339
       339
       +
               metadata = UserMetadata(

     

       340
       340
       +
                   username="testuser",

     

       341
       341
       +
                   directory="testuser",

     

       342
       342
       +
                   created=datetime.now(),

     

       343
       343
       +
                   last_updated=datetime.now(),

     

       344
       344
       +
                   entry_count=5,

     

       345
       345
       +
               )

     

       346
       346
       +
               

     

       347
       347
       +
               original_count = metadata.entry_count

     

       348
       348
       +
               original_time = metadata.last_updated

     

       349
       349
       +
               

     

       350
       350
       +
               metadata.increment_entry_count(3)

     

       351
       351
       +
               

     

       352
       352
       +
               assert metadata.entry_count == original_count + 3

     

       353
       353
       +
               assert metadata.last_updated > original_time