1
0
Fork 0
mirror of https://codeberg.org/Reuh/feather.git synced 2025-10-27 18:19:32 +00:00

Compare commits

...

12 commits
v1.0.0 ... main

8 changed files with 150 additions and 88 deletions

View file

@ -12,9 +12,9 @@ That's right, using an innovative technology known as "plain files", Feather all
![Reading an article](images/read.gif) ![Reading an article](images/read.gif)
Note however that Feather is meant to be used alongside a RSS reader server, meaning: Note that Feather is intended to be used alongside a RSS reader server, meaning:
* no grabbing feeds on its own, requiring a server to connect to :( * no grabbing feeds on its own, a separate feed reader server is required :(
* no feed management interface; Feather only support reading articles and marking them as read/unread :( * no feed management interface; Feather only support reading articles and marking them as read/unread :(
## Usage ## Usage
@ -25,57 +25,70 @@ All demonstrations are done under Linux with GNOME Files, but a file manager is
#### Navigating articles #### Navigating articles
Feed categories are directories, and each file in these directories is an article. Feed categories are directories, and each file in a category directory is an article.
![Browsing articles](images/browse.gif) ![Browsing articles and categories by navigating directories](images/browse.gif)
#### Sorting articles #### Sorting articles
An article file's modification time is the article publication time; you can sort by publication date by sorting the files by modification time, or sort by feed by sorting by filename. Each article file's modification time is set to the article publication time; you can sort by publication date by sorting the files by modification time, or sort by feed by sorting by filename.
![Sorting articles](images/sort.gif) ![Sorting articles in a category by sorting files](images/sort.gif)
#### Searching #### Searching
![Searching articles](images/search.gif) ![Searching articles in a category using filename search](images/search.gif)
Tip: if you have nested categories, search "html" to list all the articles in the category and its sub-categories in the same view. Tip: if you have nested categories, search "html" to list all the articles in the category and its sub-categories in the same view.
### Marking articles as read ### Marking articles as read
Deleting an article will toggle their read status (will take effect on the next synchronization to the server). Deleting an article file will mark them as read (will take effect on the next synchronization to the server).
![Marking an article as read](images/markasread.gif) ![Marking an article as read by deleting the article file](images/markasread.gif)
#### Handling read articles #### Handling read articles
Surprisingly, the read articles can be found in the trash. If you restore them before the next synchronization, it's be as if nothing happened. However, restoring the file after synchronization will not work if you want to mark the article as unread. The now read articles can (surprisingly) be found in the trash. After marking an article as read, there is a grace period (by default 3 days) during which you can mark read articles as unread again by restoring their files from the trash.
Instead, if you want Feather to also track read articles, you could add to your configuration file: #### Reading read articles
If you want to re-read your favorites articles directly in the Feather reader directory, you can configure Feather to write articles files for read articles too:
```toml ```toml
# Grab both read and unread articles into the local directory [html]
server.only_sync_unread_articles = false # Write article HTML files for read articles
write_read_articles = true
# Add a checkmark in the article filename indicating the read status # Add a checkmark in the article filename indicating the read status
html.filename_template = "{% if unread %}☐{% else %}☑{% endif %} [{{ feed_title }}]\t{{ title }} ({{ published }}).html" filename_template = "{% if unread %}☐{% else %}☑{% endif %} [{{ feed_title }}]\t{{ title }} ({{ published }}).html"
``` ```
Now both read and unread articles will be stored in the Feather reader directory, and if you delete a read article file, the article will be marked as unread and the file will be recreated during the next synchronization. Now both read and unread articles will be stored in the Feather reader directory, and after marking an article file as read by deleting it, Feather will regenerate the file on the next synchronization (but marked as read this time).
![Marking an article as unread](images/markunread.gif) Note that this also change the mark-as-unread behavior: since it is no longer possible to restore from the trash because the file is automatically recreated, marking an item as unread is done in the same way as mark-as-read, i.e. by deleting the file of a read article.
### Updating with the server ![Marking an article as unread by deleting the article file](images/markunread.gif)
Run `feather sync` to synchronize all local data with the server. The synchronization is done in two parts: By default, Feather will only grab unread articles from the server, so the read articles you have access to locally are only the articles kept for the 3 days grace period after marking them as read (see the [handling read articles chapter](#handling-read-articles)). If you want to have access to _all_ articles from the server, you can add to your configuration:
- `feather sync-up` which upload local changes to the server (e.g. update read status of local articles); ```toml
- `feather sync-down` which download all articles from the server into the local state. This might be a lot of data depending on how many articles you have on the server. [server]
# Grab both read and unread articles from the server
only_sync_unread_articles = false
```
### Synchronizing with the server
Run `feather sync` to synchronize all local data with the server. The synchronization is done in two parts, which you can perform separately using:
- `feather sync-up`, which upload local changes to the server (e.g. update read status of local articles);
- `feather sync-down`, which download & update all articles from the server into the local state. This might be a lot of network requests, depending on how many articles you have on the server.
If you don't want to bother running `feather sync` manually, you can also start the Feather update daemon using `feather daemon`. The daemon will periodically call `sync-up` and `sync-down` for as long as it runs. If you don't want to bother running `feather sync` manually, you can also start the Feather update daemon using `feather daemon`. The daemon will periodically call `sync-up` and `sync-down` for as long as it runs.
#### If you are offline #### If you are offline
Nothing change, but all the synchronization commands will wait until the server become reachable again. Nothing changes, but all the synchronization commands will wait until the server become reachable again.
### Configuration ### Configuration
@ -89,7 +102,7 @@ If you changed to another remote server or if you somehow messed up your local f
Since, as the kids say, everything is a file, Feather can be easily integrated with any other program which operate on files (and there's a lot of those). A couple examples: Since, as the kids say, everything is a file, Feather can be easily integrated with any other program which operate on files (and there's a lot of those). A couple examples:
- you can use Syncthing to synchronize the reader directory with other computer without having to install Feather on each one. You'd still have to install Syncthing on each one, but if for some unknown reason you prefer installing Syncthing to Feather, it works. Note that Syncthing real-time change detection can sometime miss changes when there's a lot of small files like with Feather, so you may want to reduce the _Full Rescan Interval_ in the share settings unless you're fine with some of your feeds taking a whole hour to update; - you can use Syncthing to synchronize the reader directory with other computer without having to install Feather on each device. You'd still have to install Syncthing on each one, but if for some unknown reason you prefer installing Syncthing to Feather, it works. Note that Syncthing real-time change detection can sometime miss changes when there's a lot of small files like with Feather, so you may want to reduce the _Full Rescan Interval_ in the share settings unless you're fine with some of your feeds taking a whole hour to update;
- you can process your feeds easily with regular scripts: `find reader/ -iname 'trump' -delete` will mark all feeds containing "Trump" in their filename as read; which I think is easier than messing around with your feed reader API or convoluted filtering rules directly, but that's just my opinion. - you can process your feeds easily with regular scripts: `find reader/ -iname 'trump' -delete` will mark all feeds containing "Trump" in their filename as read; which I think is easier than messing around with your feed reader API or convoluted filtering rules directly, but that's just my opinion.
## Installation ## Installation
@ -98,7 +111,7 @@ Since, as the kids say, everything is a file, Feather can be easily integrated w
If use Docker or Podman, an image is available at `codeberg.org/reuh/feather:1`. If use Docker or Podman, an image is available at `codeberg.org/reuh/feather:1`.
For example, to start a new container with the Feather daemon running, using the `config.yml` (see [basic configuration](#basic-configuration)) in the current directory and exposing the feeds in the `reader` directory: For example, to start a new container running the Feather daemon, using the configuration file `config.yml` (see [basic configuration](#basic-configuration)) from the current directory, and putting the articles in the `reader` directory:
```sh ```sh
docker run -d -v ./config.toml:/feather/config.toml:ro -v ./reader:/feather/reader -v feather-data:/feather/data --name feather codeberg.org/reuh/feather:1 daemon docker run -d -v ./config.toml:/feather/config.toml:ro -v ./reader:/feather/reader -v feather-data:/feather/data --name feather codeberg.org/reuh/feather:1 daemon
@ -108,7 +121,7 @@ Instead of using a configuration file, you may also use environment variables; s
If you need to run Feather commands in a running container, run for example `docker exec feather feather regenerate`. Otherwise you could also start a new container as described above but using a command other than `daemon`. If you need to run Feather commands in a running container, run for example `docker exec feather feather regenerate`. Otherwise you could also start a new container as described above but using a command other than `daemon`.
If you're wondering how monstrous the resources required by Feather are, with my ~600 feeds on my Tiny Tiny RSS, Feather stays at ~50MB RAM usage and take ~4KB of disk space per article. The docker image is ~120MB, which does not fit on a floppy disk but does easily on a CD. If you're wondering how monstrous the resources required by Feather are, with my ~600 feeds on my Tiny Tiny RSS server, Feather stays at ~50MB RAM usage and take ~4KB of disk space per article. The docker image is ~120MB, which does not fit on a floppy disk but does easily on a CD.
#### Quadlet #### Quadlet
@ -152,7 +165,7 @@ systemctl --user start feather
Feather should be able to run on anything that can run Python 3.12 or newer (you might need to change `html.max_filename_length` and `html.filename_replacement` in the configuration if your filesystem has exotic limitations). Feather should be able to run on anything that can run Python 3.12 or newer (you might need to change `html.max_filename_length` and `html.filename_replacement` in the configuration if your filesystem has exotic limitations).
Once you have Python installed, download this repository and pip it up by running `pip install` inside it. You should be able to run `feather sync` or whatever command you want to run according to the [usage chapter](#usage). Although I personally use uv, so I'd just run `uv sync` and then `uv run feather sync` to run Feather. Once you have Python installed, download this repository and pip it up by running `pip install` inside it. You should be able to run `feather sync` or whatever command you want to run according to the [usage chapter](#synchronizing-with-the-server). If you use uv, skip the pip and run directly `uv run feather sync`.
### Basic configuration ### Basic configuration
@ -192,28 +205,21 @@ I mean nobody asked them yet but I think I have a pretty good grasp on what peop
### Why? ### Why?
I rely on RSS a lot to keep in touch with outside my room, and I've been using Tiny Tiny RSS to do that for years at this point. Unfortunately, the historical developer decided to stop development this October 2025, and after trying other RSS readers I was unsatisfied and decided to make my own. As I was contemplating how to design the RSS reader of my dreams, I had a epiphany: a RSS reader is nothing more than directories and files. This was great because it meant I didn't have to make a GUI, as like most people, I find working more than necessary generally unpleasant. But then, it turned out that as far as personal effort go, something even better happened: the Tiny Tiny RSS community is continuing its development. Relieved but at the same time dejected that my epiphany went to waste, I thus decided to make only the directories and files part of my RSS reader, and let Tiny Tiny RSS do the rest. And here we are. I rely on RSS a lot to keep in touch with outside my room, and I've been using Tiny Tiny RSS to do that for years at this point. Unfortunately, the historical developer decided to stop development this October 2025, and after trying other RSS readers I was unsatisfied and decided to make my own. As I was contemplating how to design the RSS reader of my dreams, I had a epiphany: a RSS reader is nothing more than directories and files. This was great because it meant I didn't have to make a GUI, and like most people, I find working more than necessary generally unpleasant. But then, it turned out that as far as personal effort go, something even better happened: the Tiny Tiny RSS community is continuing development! Relieved but at the same time dejected that my epiphany went to waste, I thus decided to make only the directories and files part of my RSS reader, and let Tiny Tiny RSS do the rest. And here we are.
### Why the name "feather"? ### Why the name "feather"?
My mom taught me that if all my friends jump off a bridge I should too, and since most popular open-source projects are named after a common English noun I decided to do the same. If you're looking for a set of icons, a JavaScript framework, ML libraries, data analytics libraries, a game server, a smartphone application, or a lot of other things that aren't a RSS reader client, I regret to inform you that you have unfortunately found the wrong Feather. My mom taught me that if all my friends jump off a bridge I should too, and since most popular open-source projects are named after a common English noun these days I decided to do the same. If you're looking for a set of icons, a JavaScript framework, ML libraries, data analytics libraries, a game server, a smartphone application, or a lot of other things that aren't a RSS reader client, I regret to inform you that you have unfortunately found the wrong Feather.
### What should I eat tonight? ### What should I eat tonight?
Spaghetti alla carbonara is often a safe choice; even if you substitutes all of the ingredients you'll likely still end up with something decent. It still counts as carbonara if you use cream and mushrooms, trust me, I know how to say hello in Italian. Spaghetti alla carbonara is often a safe choice; even if you substitute all the ingredients you'll likely still end up with something decent. It still counts as carbonara if you use cream and mushrooms, trust me, I know how to say hello in Italian.
## Future improvements ## Future improvements
While I mostly started this project for fun, it ended up actually quite usable, so I'm at least going to maintain it. As for new features, unless an incredible idea comes I'd like to keep this project simple (try your luck in the issue tracker!). But still, here's still some things that I wrote down during development that _may_ be done at some point: While I mostly started this project for fun, it ended up actually quite usable, so I'm at least going to maintain it. As for new features, unless an incredible idea comes I'd like to keep this project simple (but if you think it falls within the scope of a "file-based RSS reader client", try your luck in the issue tracker!). But still, here's still some things that I wrote down during development that _may_ be done at some point:
- [ ] Store & expose article attachments in the templates - [ ] Store & expose article attachments in the templates
- [ ] Partial synchronization (using since_id for ttrss (article updates?) and start_time for googlereader) - [ ] Partial synchronization (using since_id for ttrss (article updates?) and start_time for googlereader)
- [ ] Think of a third one, two is a bit sad - [ ] Think of a third future improvement, two is a bit sad
# TODO before publishing
- [x] Write documentation
- [ ] Re-read
- [x] Tag v1.0 in git & container
- [ ] Publish

View file

@ -1,6 +1,6 @@
[project] [project]
name = "feather" name = "feather"
version = "1.0.0" version = "1.1.1"
authors = [ { name = 'Étienne "Reuh" Fildadut' } ] authors = [ { name = 'Étienne "Reuh" Fildadut' } ]
description = "file-based RSS reader client" description = "file-based RSS reader client"
readme = "README.md" readme = "README.md"

View file

@ -6,6 +6,7 @@ from asyncio import Event
from typing import Iterable from typing import Iterable
from watchfiles import awatch from watchfiles import awatch
from pathlib import Path from pathlib import Path
from datetime import datetime
from feather.config import Config from feather.config import Config
from feather.client import GReaderSession, TTRSession, ClientSession, Article, ArticleId from feather.client import GReaderSession, TTRSession, ClientSession, Article, ArticleId
@ -82,13 +83,16 @@ class FeatherApp:
to_mark_as_read = [] to_mark_as_read = []
to_mark_as_unread = [] to_mark_as_unread = []
for article in self.iter_articles(): for article in self.iter_articles():
if not article.has_html(): has_html = article.has_html()
if article.unread: if article.unread and not has_html:
to_mark_as_read.append(article) to_mark_as_read.append(article)
marked_as_read += 1 marked_as_read += 1
else: elif not article.unread and (
to_mark_as_unread.append(article) (config.write_read_articles and not has_html)
marked_as_unread += 1 or (not config.write_read_articles and has_html)
):
to_mark_as_unread.append(article)
marked_as_unread += 1
if len(to_mark_as_read) == len(to_mark_as_unread) == 0: if len(to_mark_as_read) == len(to_mark_as_unread) == 0:
return # nothing to do return # nothing to do
@ -106,13 +110,10 @@ class FeatherApp:
to_mark_as_unread_id[i : i + config.articles_per_query], False to_mark_as_unread_id[i : i + config.articles_per_query], False
) )
# regenerate/delete local file with new read/unread state # regenerate local file with new read/unread state
for article in to_mark_as_read: for article in to_mark_as_read:
article.unread = False article.unread = False
if config.only_sync_unread_articles: article.regenerate()
article.delete()
else:
article.regenerate()
for article in to_mark_as_unread: for article in to_mark_as_unread:
article.unread = True article.unread = True
article.regenerate() article.regenerate()
@ -127,7 +128,7 @@ class FeatherApp:
print("Synchronizing from server...") print("Synchronizing from server...")
new_articles, updated_articles = 0, 0 new_articles, updated_articles = 0, 0
grabbed_article_paths: set[ArticleId] = set() grabbed_article_ids: set[ArticleId] = set()
categories = client_session.list_categories() categories = client_session.list_categories()
for category in categories: for category in categories:
@ -147,7 +148,7 @@ class FeatherApp:
remaining = False remaining = False
for article in articles: for article in articles:
grabbed_article_paths.add(article.id) grabbed_article_ids.add(article.id)
json_path = article.json_path json_path = article.json_path
if not json_path.exists(): if not json_path.exists():
article.write() article.write()
@ -159,12 +160,26 @@ class FeatherApp:
article.write() article.write()
updated_articles += 1 updated_articles += 1
# Remove articles that we didn't get from the server but are in the JSON directory # Remove or mark-as-read articles that we didn't get from the server but are in the JSON directory
removed_articles = 0 removed_articles = 0
article_cutoff_timestamp = (
datetime.now().timestamp() - config.keep_read_articles_for
)
for article in self.iter_articles(): for article in self.iter_articles():
if article.id not in grabbed_article_paths: if article.id not in grabbed_article_ids:
article.delete() # we only sync unread: articles we didn't get from the server were read or purged
removed_articles += 1 if config.only_sync_unread_articles:
if article.last_write < article_cutoff_timestamp:
article.delete()
removed_articles += 1
elif article.unread:
article.unread = False
article.regenerate()
updated_articles += 1
# we sync all articles: articles we didn't get from the server were purged
else:
article.delete()
removed_articles += 1
print( print(
f"Synchronization successful ({new_articles} new articles, {updated_articles} updated, {removed_articles} removed)" f"Synchronization successful ({new_articles} new articles, {updated_articles} updated, {removed_articles} removed)"
@ -260,6 +275,7 @@ class FeatherApp:
"""Regenerate all local files using local data only""" """Regenerate all local files using local data only"""
for article in self.iter_articles(): for article in self.iter_articles():
article.regenerate() article.regenerate()
self.remove_empty_categories()
def clear_data(self): def clear_data(self):
"""Delete all local data""" """Delete all local data"""

View file

@ -4,7 +4,7 @@ from __future__ import annotations
import re import re
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from ttrss.client import TTRClient from ttrss.client import TTRClient, Headline
import google_reader import google_reader
from feather.config import Config from feather.config import Config
@ -146,12 +146,21 @@ class GReaderArticle(Article):
# several API references I've seen didn't mention canonical, but alternate seems to also be the article link (?) and should be an ok fallback # several API references I've seen didn't mention canonical, but alternate seems to also be the article link (?) and should be an ok fallback
self.article_url = item_content.alternate[0].href self.article_url = item_content.alternate[0].href
self._compute_json_path() self.json_path = self._get_json_path()
## Tiny Tiny RSS API ## ## Tiny Tiny RSS API ##
# Monkey patch Headline.__init__ to skip timestamp to datetime conversion
# Articles may have a negative timestamp and Python's datetime.fromtimestamp doesn't like that, so instead we keep the timestamp and deal with the issue in data.py/format_datetime
def Headline_init(self, attr, client):
super(Headline, self).__init__(attr, client)
Headline.__init__ = Headline_init
class TTRSession(ClientSession): class TTRSession(ClientSession):
"""Tiny Tiny RSS API client""" """Tiny Tiny RSS API client"""
@ -234,8 +243,8 @@ class TTRArticle(Article):
self.unread = article.unread self.unread = article.unread
self.title = article.title self.title = article.title
self.published = article.updated.timestamp() self.published = article.updated
self.updated = article.updated.timestamp() self.updated = article.updated
self.author = article.author self.author = article.author
self.summary = article.excerpt self.summary = article.excerpt
self.content = article.content self.content = article.content
@ -248,4 +257,4 @@ class TTRArticle(Article):
self.language = article.lang self.language = article.lang
self.image_url = article.flavor_image self.image_url = article.flavor_image
self._compute_json_path() self.json_path = self._get_json_path()

View file

@ -6,6 +6,7 @@
[server] [server]
# (Required) Server API to use. Either "googlereader" for the Google Reader API (FreshRSS, Miniflux, etc.) or "ttrss" for the TinyTiny-RSS API. # (Required) Server API to use. Either "googlereader" for the Google Reader API (FreshRSS, Miniflux, etc.) or "ttrss" for the TinyTiny-RSS API.
# The Google Reader API do not support nested categories. # The Google Reader API do not support nested categories.
# Can be set through the environment variable SERVER_API.
api = "googlereader" api = "googlereader"
# (Required) URL of your server's Google Reader API endpoint # (Required) URL of your server's Google Reader API endpoint
# Can be set through the environment variable SERVER_URL. # Can be set through the environment variable SERVER_URL.
@ -23,9 +24,17 @@ password = "password"
# Set to 0 to let Feather choose (200 for ttrss, 1000 for googlereader). # Set to 0 to let Feather choose (200 for ttrss, 1000 for googlereader).
# Can be set through the environment variable SERVER_ARTICLES_PER_REQUEST. # Can be set through the environment variable SERVER_ARTICLES_PER_REQUEST.
articles_per_request = 0 articles_per_request = 0
# Set to true to only sync unread articles; Feather will not retrieve or store any read article. # Set to true to only sync unread articles; Feather will not retrieve any read article from the server.
# If set to false, Feather will download ALL articles from the server, read and unread, on each synchronization. This might be a lot of data depending on how many read articles your server keeps. If you only want to keep recent read articles, look at the keep_read_articles_for settings below.
# Can be set through the environment variable SERVER_ONLY_SYNC_UNREAD_ARTICLES. # Can be set through the environment variable SERVER_ONLY_SYNC_UNREAD_ARTICLES.
only_sync_unread_articles = true only_sync_unread_articles = true
# How long in seconds to keep read articles in the local storage before deleting them.
# Once an article is removed, Feather can no longer:
# - mark it as unread when its article file is restored from the trash;
# - generate articles files for read articles if html.write_read_articles = true.
# If only_sync_unread_articles = false, this does nothing (since Feather always retrieve all read articles from the server).
# Can be set through the environment variable SERVER_KEEP_READ_ARTICLES_FOR.
keep_read_articles_for = 259200
[directories] [directories]
# Data directory: path where the internal Feather data will be stored. # Data directory: path where the internal Feather data will be stored.
@ -36,6 +45,12 @@ data = "data"
reader = "reader" reader = "reader"
[html] [html]
# If set to true, Feather will also generate articles files for read articles.
# The the mark-as-unread behavior will change depending on this value:
# - if false, marking an article as unread requires its file to be recreated/restored from the trash;
# - if true, marking an article as unread requires deleting its article file (same as mark-as-read).
# Can be set through the environment variable HTML_WRITE_READ_ARTICLES.
write_read_articles = false
# Template used for generating article HTML files. All templates are Jinja2 templates. # Template used for generating article HTML files. All templates are Jinja2 templates.
# Available fields: # Available fields:
# - id: article id (int | str) # - id: article id (int | str)
@ -49,7 +64,7 @@ reader = "reader"
# - feed_title: feed title (str) # - feed_title: feed title (str)
# - feed_url: feed URL (str) # - feed_url: feed URL (str)
# - feed_icon_url: feed icon URL (str) # - feed_icon_url: feed icon URL (str)
# - feed_order: feed display order, starting from 1 (0 if unknown) (int) # - feed_order: feed display order, starting from 1 (0 if unknown, only available for ttrss) (int)
# - article_url: article URL (str) # - article_url: article URL (str)
# - comments_url: article comments URL (str) # - comments_url: article comments URL (str)
# - language: article language (str) # - language: article language (str)
@ -70,7 +85,7 @@ article_template = '''
<body style="background-color:black; color:white; font-family:sans-serif; overflow-x:hidden;"> <body style="background-color:black; color:white; font-family:sans-serif; overflow-x:hidden;">
<style> <style>
a {color:palevioletred; text-decoration:none;} a {color:palevioletred; text-decoration:none;}
.feather-content img {display:block; margin-left:50%; transform:translateX(-50%); max-width:100vw;} .feather-content img, .feather-content video {display:block; margin-left:50%; transform:translateX(-50%); max-width:100vw;}
</style> </style>
<article style="max-width:min(60rem,100%); margin:auto;"> <article style="max-width:min(60rem,100%); margin:auto;">
<p style="display:flex; flex-direction:row; justify-content:space-between;"> <p style="display:flex; flex-direction:row; justify-content:space-between;">
@ -113,7 +128,7 @@ filename_template = "[{{ feed_title }}]\t{{ title }} ({{ published }}).html"
# - id: category id (str | int) # - id: category id (str | int)
# - title: category name (str) # - title: category name (str)
# - parents: list of parent categories (list[Category]) # - parents: list of parent categories (list[Category])
# - order: category display order, starting from 1 (0 if unknown) (int) # - order: category display order, starting from 1 (0 if unknown, only available for ttrss) (int)
# If empty, no directory will be created. # If empty, no directory will be created.
# Can be set through the environment variable HTML_CATEGORY_TEMPLATE. # Can be set through the environment variable HTML_CATEGORY_TEMPLATE.
category_template = "{% if order %}{{ '%02d' % order }} {% endif %}{{ title }}" category_template = "{% if order %}{{ '%02d' % order }} {% endif %}{{ title }}"
@ -124,6 +139,7 @@ hide_empty_categories = true
# Can be set through the environment variable HTML_MAX_FILENAME_LENGTH. # Can be set through the environment variable HTML_MAX_FILENAME_LENGTH.
max_filename_length = 250 max_filename_length = 250
# Table mapping characters to what they will be replaced with in filenames. Useful to remove/replace characters that are not allowed in filename by your filesystem. The default should be fine for most Unix filesystems. # Table mapping characters to what they will be replaced with in filenames. Useful to remove/replace characters that are not allowed in filename by your filesystem. The default should be fine for most Unix filesystems.
# Can not be set through environment variables, sorry!
filename_replacement = { "/" = "", "\u0000" = "" } filename_replacement = { "/" = "", "\u0000" = "" }
[datetime] [datetime]

View file

@ -61,10 +61,14 @@ class Config:
self.only_sync_unread_articles: bool = bool( self.only_sync_unread_articles: bool = bool(
get_config("server", "only_sync_unread_articles") get_config("server", "only_sync_unread_articles")
) )
self.keep_read_articles_for: float = float(
get_config("server", "keep_read_articles_for")
)
self.timezone: ZoneInfo = ZoneInfo(str(get_config("datetime", "timezone"))) self.timezone: ZoneInfo = ZoneInfo(str(get_config("datetime", "timezone")))
self.time_format: str = str(get_config("datetime", "format")) self.time_format: str = str(get_config("datetime", "format"))
self.write_read_articles: bool = bool(get_config("html", "write_read_articles"))
self.article_template: Template = Template( self.article_template: Template = Template(
str(get_config("html", "article_template")), autoescape=True str(get_config("html", "article_template")), autoescape=True
) )

View file

@ -5,7 +5,7 @@ from __future__ import annotations
import os import os
import json import json
from abc import ABC from abc import ABC
from datetime import datetime from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
from hashlib import sha256 from hashlib import sha256
from tempfile import NamedTemporaryFile from tempfile import NamedTemporaryFile
@ -16,7 +16,7 @@ from feather.config import Config
def sanitize_filename( def sanitize_filename(
config: Config, filename: str, insert_before_suffix: str = "" config: Config, filename: str, insert_before_suffix: str = ""
) -> str: ) -> str:
"""Escape invalid caracters and truncate the filename as per the configuration. """Escape invalid characters and truncate the filename as per the configuration.
This operates on a single filename, not a path. This operates on a single filename, not a path.
(insert_before_suffix will be inserted between the stem and suffix, and is assumed to not need escaping).""" (insert_before_suffix will be inserted between the stem and suffix, and is assumed to not need escaping)."""
filename = filename.translate(config.filename_translation) filename = filename.translate(config.filename_translation)
@ -38,11 +38,15 @@ def sanitize_filename(
return filename[:cutoff] + "" + insert_before_suffix + suffix return filename[:cutoff] + "" + insert_before_suffix + suffix
def format_datetime(config: Config, timestamp: int) -> str: def format_datetime(config: Config, timestamp: float) -> str:
"""Format a timestamp according to the configuraiton.""" """Format a timestamp according to the configuration."""
return datetime.fromtimestamp(timestamp, config.timezone).strftime( if timestamp < 0:
config.time_format date = datetime(1970, 1, 1, tzinfo=config.timezone) + timedelta(
) seconds=timestamp
)
else:
date = datetime.fromtimestamp(timestamp, config.timezone)
return date.strftime(config.time_format)
def atomic_write(path: Path, content: str): def atomic_write(path: Path, content: str):
@ -99,8 +103,8 @@ class Article(ABC):
# with default value # with default value
unread: bool = True # if the article is unread unread: bool = True # if the article is unread
title: str = "" # article title title: str = "" # article title
published: int = 0 # article publication time (timestamp) published: float = 0.0 # article publication time (timestamp)
updated: int = 0 # article update time (timestamp) updated: float = 0.0 # article update time (timestamp)
author: str = "" # article author author: str = "" # article author
summary: str = "" # article summary (HTML) summary: str = "" # article summary (HTML)
content: str = "" # article content (HTML) content: str = "" # article content (HTML)
@ -112,6 +116,7 @@ class Article(ABC):
comments_url: str = "" # article comments URL comments_url: str = "" # article comments URL
language: str = "" # article language language: str = "" # article language
image_url: str = "" # article main image image_url: str = "" # article main image
last_write: float = 0.0 # last time this article file was written (timestamp)
def _hash_id(self): def _hash_id(self):
return sha256(str(self.id).encode("utf-8")).hexdigest() return sha256(str(self.id).encode("utf-8")).hexdigest()
@ -168,13 +173,13 @@ class Article(ABC):
d["category"] = self.category.asdict() d["category"] = self.category.asdict()
return d return d
def _compute_json_path(self): def _get_json_path(self) -> Path:
self.json_path = self.config.json_root / f"{self._hash_id()}.json" return self.config.json_root / f"{self._hash_id()}.json"
def _write_json(self, recompute_path=False): def _write_json(self, recompute_path=False):
"""Write the JSON file associated with this article. Error if it already exists.""" """Write the JSON file associated with this article. Error if it already exists."""
if recompute_path: if recompute_path:
self._compute_json_path() self.json_path = self._get_json_path()
stored_fields = ( stored_fields = (
"id", "id",
"unread", "unread",
@ -193,6 +198,7 @@ class Article(ABC):
"language", "language",
"image_url", "image_url",
"html_path", "html_path",
"last_write",
) )
article_json = {field: getattr(self, field) for field in stored_fields} article_json = {field: getattr(self, field) for field in stored_fields}
article_json["category"] = self.category.asdict() article_json["category"] = self.category.asdict()
@ -225,13 +231,16 @@ class Article(ABC):
html_path, config.article_template.render(self._get_template_dict()) html_path, config.article_template.render(self._get_template_dict())
) )
# set accessed date to update time, modified to publication time # set accessed date to update time, modified to publication time
os.utime(html_path, (max(self.published, self.updated), self.published)) os.utime(html_path, (max(int(self.published), int(self.updated)), int(self.published)))
def _delete_html(self, missing_ok=False): def _delete_html(self, missing_ok=False):
"""Delete the HTML file associated with this article.""" """Delete the HTML file associated with this article."""
# Delete a HTML file for a JSON object # Delete a HTML file for a JSON object
html_path = self.config.html_root / self.html_path if self.html_path is None:
html_path.unlink(missing_ok=missing_ok) return
else:
html_path = self.config.html_root / self.html_path
html_path.unlink(missing_ok=missing_ok)
def has_html(self) -> bool: def has_html(self) -> bool:
"""Check if the HTML file associated with the article exists on disk.""" """Check if the HTML file associated with the article exists on disk."""
@ -247,13 +256,15 @@ class Article(ABC):
def write(self, recompute_paths=False): def write(self, recompute_paths=False):
"""Write all the files associated with this article to disk.""" """Write all the files associated with this article to disk."""
try: self.last_write = datetime.now().timestamp()
self._write_html(recompute_path=recompute_paths) if self.unread or self.config.write_read_articles:
except FileExistsError: try:
raise self._write_html(recompute_path=recompute_paths)
except: except FileExistsError:
self._delete_html(missing_ok=True) raise
raise except:
self._delete_html(missing_ok=True)
raise
try: try:
self._write_json(recompute_path=recompute_paths) self._write_json(recompute_path=recompute_paths)
except FileExistsError: except FileExistsError:

2
uv.lock generated
View file

@ -69,7 +69,7 @@ wheels = [
[[package]] [[package]]
name = "feather" name = "feather"
version = "1.0.0" version = "1.1.1"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "google-reader" }, { name = "google-reader" },