Introducing reitti: a selfhosted alternative to Google Timeline (github.com)
from danielgraf@discuss.tchncs.de to selfhosted@lemmy.world on 03 Jul 04:40
https://discuss.tchncs.de/post/39888199

Hey everyone!

I’m excited to introduce Reitti, a location tracking and analysis application designed to help you gain insights about your movement patterns and significant places—all while keeping your data private on your own server.

Core Capabilities:

Photo Integration:

Data Import Options:

Customization:

Self-hosting:

Reitti is still early in development but has already developed extensive capabilities. I’d love to hear your feedback and answer any questions to tailor Reitti to meet the community’s needs.

Hope this sparks some interest!

Daniel

#selfhosted

threaded - newest

otter@lemmy.ca on 03 Jul 04:49 next collapse

Cool!

I love the UI for this one, it’s unique compared to the others

danielgraf@discuss.tchncs.de on 03 Jul 05:40 collapse

Thanks otter, I tried to have a historical look. Like going to the past and revive some memories. Hope it worked out. I am actually pretty happy with it.

Lost_My_Mind@lemmy.world on 03 Jul 04:57 next collapse

Man. I have no use for this. I know where I go. I go to work. And then the gym. Almost every day. Because I work a lot.

It’s like my boss always says…“BACK TO WORK, DICKHEAD!”

Zwuzelmaus@feddit.org on 03 Jul 05:00 next collapse

I still have a phone containing Google, and cannot change this situation (maybe with my next phone). Usually, I switch off location services very often and I avoid such tracking apps because all my data goes to Google then.

Would it be advisable to use this at all before I get rid of Google?

danielgraf@discuss.tchncs.de on 03 Jul 05:13 collapse

Hello @Zwuzelmaus@feddit.org, i usually use GPSLogger for Android to track my location during the day and this periodically sends the data to reitti whenever i am back at home. I have no idea if you switch off location services what happens on the GPSLogger side of the chain. If it still be able to access GPS I see no problem, if not than this sadly will break the usecase for reitti.

It relies on a consistent GPS tracking data to be able to do its thing

Zwuzelmaus@feddit.org on 03 Jul 07:22 collapse

if you switch off location services

…then no app gets a location anymore.

Sibbo@sopuli.xyz on 03 Jul 05:02 next collapse

This looks amazing, and I will try it out once it has reasonable NixOS support.

I especially like the immich integration.

Does it increase battery consumption of the phone a lot?

danielgraf@discuss.tchncs.de on 03 Jul 05:16 next collapse

Reitti on its own does not run on your phone. It relies on external apps to track your GPS location like GPSLogger or Owntracks. They will increase the battery consumption but I think it is bearable. I personally have setup GPSLogger to fetch every 30 seconds a GPS location and then sends it every hour to reitti. With that i do not see that more power usage. Currently starting to test how the power drain is with a 15 seconds interval.

smiletolerantly@awful.systems on 03 Jul 05:18 next collapse

Better open a package request (or pull request :D) then 😄

beerclue@lemmy.world on 03 Jul 05:53 collapse

May I ask what you mean by NixOS support? There’s a docker compose you could use in their repo…

Sibbo@sopuli.xyz on 03 Jul 08:01 collapse

But then I’d have to combine that with existing services myself. NixOS ensures that there is only one postgres running for example, such that not every service starts its own postgres instance.

For immich, for example the following is enough for a most basic configuration:

services.immich.enable = true;
services.immich.port = 2283;
beerclue@lemmy.world on 03 Jul 09:20 collapse

I see what you mean, interesting. Didn’t really look at NixOS as a server os. I personally prefer using multiple compose files (in the process of migrating to k8s). I share resources too, like in your example, I just point to the existing DB instance, not create a new one for each new service.

MrQuallzin@lemmy.world on 03 Jul 05:13 next collapse

This is really awesome! I was just about to start looking for something like this, so great timing. Going to get this up on my Unraid server tomorrow and play around with it

Atropos@lemmy.world on 03 Jul 06:00 next collapse

This is really neat, I’m going to spin it up tomorrow!

danielgraf@discuss.tchncs.de on 03 Jul 06:08 collapse

Let me know how it worked out. I am deploying it to my server via docker without an problems, but maybe they are some quirks i missed. :D

Atropos@lemmy.world on 03 Jul 06:24 collapse

Of course, happy to do so. I’ll report back with my experience when I have it working!

Zagorath@aussie.zone on 03 Jul 06:22 next collapse

Fuck yeah this is awesome! The detail of Immich integration is just the icing on top of an awesome cake!

How demanding is it on server resources? Am I likely to be able to run it on an old Raspberry Pi that’s also running a couple of other relatively light tasks? How much storage does it end up using over time? I’m probably going to try and get it running either on my Pi or my Synology NAS, though the latter has had issues with Docker containers in the past depending on the container’s dependencies…

danielgraf@discuss.tchncs.de on 03 Jul 06:47 next collapse

I have no clue if a raspberry will handle it. There a a couple of services involved to make it fast, but they are then another burden like RabbitMQ. Which make ingesting data instantaneous but you need extra processing power to handle the queues. It all comes with a tradeoff.

For size, there is mainly the PostGIS DB. I just checked and my db is around 800 MB for roughtly 8 1/2 Years of data.

Photon (the reverse geocode enabled in the compose file) is another beast. For Germany it takes 14 GB of storage while running, if you let PARALLELL updates enabled you can double that every time the index is updated. But you can remove that from the compose file and rely on external Geocoders. It is described in github.com/dedicatedcode/reitti?tab=readme-ov-fil…

Zagorath@aussie.zone on 03 Jul 09:05 collapse

Oh interesting. I’ve just read through that link, and I was assuming that something similar to the “external only” option would have been the only way it worked. More specifically, I thought it’d just store a list of historical points and display those on an OSM overlay. But it seems like even “external only” is much more involved than that.

What happens with self-hosted Photon if you specify a country, but then also visit another country? (I assume in hybrid mode it’s as simple as "use Photon in your country, use Nominatim otherwise?)

But yeah, definitely sounds like a Pi is probably not gonna cut it. I’ll have to see if my Synology can do it, or if the weird OS restrictions Synology imposes prevent it.

danielgraf@discuss.tchncs.de on 03 Jul 10:06 collapse

If you use Photon and only have your main country available, it will fallback to the configured external Geo-coding-services since Photon will not return a result then. So the order of execution is:

  • first try Photon
  • if it does not return anything, try to call one or all of the available Geo coding services.
Zagorath@aussie.zone on 03 Jul 11:47 collapse

Is that true even if you’re not in hybrid mode?

danielgraf@discuss.tchncs.de on 03 Jul 12:01 collapse

Maybe the wording is confusing in the Readme. Reitti will try to fetch the data from a configured photon instance first, if this does not return anything and you have Geocoding services configured, it will try them. There is actually no switch for hybrid mode or only local. It depends on what is configured.

Photon Only: you have only photon configured and under Settings > Gecoding you deleted or disable every available service. Hybrid Mode: Photon is configured and under Settings > Geocoding there are Services available. That es is the one I use. Having Photon with the data for Germany and all the rest is handled by Geoapify.com.

External Only: You dropped Photon from the docker-compose file and only rely on services under Settings > Geocoding

If you do not configure anything, then Reitti will skip Geocoding and only display Unknown Place.

I will update the Readme to make that clear.

Zagorath@aussie.zone on 03 Jul 12:33 collapse

If you do not configure anything, then Reitti will skip Geocoding and only display Unknown Place.

Ah ok thanks. This is what I was wondering.

Two follow-ups:

Can you specify multiple COUNTRY_CODEs? (and if so, is the method

environment:
  - COUNTRY_CODE=country_one
  - COUNTRY_CODE=country_two

or

environment:
  - COUNTRY_CODE=[country_one, country_two]

or something else?)

And is this something that can seemlessly be retroactively changed? For example, if I set COUNTRY_CODE=au and it works fine for Australia, but then I move to NZ, can I add (assuming the answer to my first question is yes) or change to COUNTRY_CODE=nz and have all the NZ locations work on the already-recorded data, even if I made that change to my configuration after I had been in NZ for a few months?

danielgraf@discuss.tchncs.de on 03 Jul 12:38 collapse

Good question, afaik you can not enter multiple countries to Photon. I was hoping it would be possible but everything i saw was it is either on country or the whole world. But maybe you can have a look here: github.com/komoot/photon That is the service we are using.

Zagorath@aussie.zone on 03 Jul 15:09 collapse

I just took a really quick look at it, but under Importing data from Nominatim it says “-country-codes allows to filter the data to be imported by country. Set this to a comma-separated list of two-letter language codes.”

That’s a different section from the Importing data from a JSON dump section, which is where it only mentions -country-code. But even that does seem to suggest it takes “all the parameters of an import from a Nominatim database”. So it seems like either the documentation for one of them is wrong, or both are lacking (because in fact both the singular and plural work).

danielgraf@discuss.tchncs.de on 03 Jul 15:43 next collapse

I think this is not exposed when running the Docker container. But let me check later when i have time what happens if i put another country in that variable

danielgraf@discuss.tchncs.de on 03 Jul 16:32 collapse

I looked at the docker image i am using in the docker-compose file and this only supports having a single country code. The actual reason can be found here: github.com/rtuszik/…/start-photon.sh#L341C5-L342C…

It is probably possible if you deploy photon on its own and then import the data somehow. But that is to much hassle for me, i think and hope that most of the use case is handled by the current solution. At least for most of the potential users. But I get the point if someone is traveling a lot between countries.

If there is enough demand I could maybe try to create a PR for the Docker image to handle multiple country codes.

acchariya@lemmy.world on 03 Jul 06:52 collapse

Looks to be a java application with a number of services running alongside- I don’t think it’s going to be lightweight to run on resource constrained devices, but sweet project anyway! (Side note, no clue how you engineers find the time to hack on things like this, I feel like I’ve got so little time to myself I cannot imagine dedicating it to a project like this)

BlueEther@no.lastname.nz on 03 Jul 06:33 next collapse

This looks good, I’m currently using dawarich and owntracks to track our families 5 week road trip around the south island of New Zealand (no.lastname.nz/post/1468113). If I can find a spare moment in our travels and a good internet connection I’ll try and spin up an image and test it out.

One thing that I would love is to be able to have multiple people displayed on a single map (my wife [kids when they have their own phones with data], brother, mother and a few close friends - we already use google maps for this)

danielgraf@discuss.tchncs.de on 03 Jul 06:51 collapse

That sounds like an awesome idea, actually the data is stored for every user separately depending on the used api token or when you login and do a data import. I will create a feature request for it.

But bear in mind, reitti is not meant to be a real-time tracking app. At the moment, data is processed every 10 minutes.

BlueEther@no.lastname.nz on 03 Jul 08:27 collapse

Real time is nice, but not a must have at least for my use case.

corsicanguppy@lemmy.ca on 03 Jul 06:56 next collapse

Docker-dependent? It looks fantastic, but I have no containers in my home-lab – and it’s based on my time managing OS security for an OS. I’m stuck living vicariously through the rest of you, so report back often.

pemptago@lemmy.ml on 03 Jul 07:08 next collapse

This looks awesome! I’d just been collect my GPS data thinking one day I’d analyze/visualize it on my own-- not expecting anyone else to make a such a comprehensive program. The UI looks really slick. I’m looking forward to testing it out. Hopefully this weekend. Thank you for this!

danielgraf@discuss.tchncs.de on 03 Jul 07:22 collapse

Let me know how it works out for you. If you have the gpx files, you can simply import them inside the settings menu.

scarilog@lemmy.world on 03 Jul 07:21 next collapse

My only concern is battery usage. Google has the advantage of OS integration, which skims location data for timeline history even when another app accesses location, which uses essentially no battery (since you would’ve been using location for that other app anyway).

But it’s awesome that a tool like this exists anyways, great work.

danielgraf@discuss.tchncs.de on 03 Jul 08:35 collapse

Thank you :)

I understand your concerns, this is something every additional app would have to deal with.

For me it is ok to have GPSLogger running all the time, I think for what it is doing it is quite easy on the battery but I do not use my phone actively that much and I am happy if it survives a day which it does.

illusionist@lemmy.zip on 03 Jul 14:15 collapse

I log my position every 5 seconds for years and gpslogger is not among any of the battery consuming processes if you use the device. If you don’t use the phone at all, it will consume more power but that’s no reason not to use it.

dan@upvote.au on 03 Jul 07:41 next collapse

I’d love to see an integration with PhotoStructure in addition to Immich.

danielgraf@discuss.tchncs.de on 03 Jul 08:18 collapse

Never heard of PhotoStructure but if they provide an API where i can search assets for a day and it also returns the exif data especially latitutde and longitude it should be pretty straight forward to implement. Feel free to add a feature request when you got time and I will have a look at it :)

nieceandtows@programming.dev on 03 Jul 07:43 next collapse

I’ve been putting off setting up an immich server. Would this do well if hosted on the same machine?

Zagorath@aussie.zone on 03 Jul 09:09 next collapse

I love that it supports multiple formats for important location as well as multiple geocoders. But that makes me wonder, would it be feasible to support multiple image libraries? There’s a bunch of different FOSS photo libraries out there. I think Nextcloud is the main other one I’ve heard about ‘in the wild’, as it were. Or is there too much bespoke Immich code in there for that to be a simple plug-and-play option?

danielgraf@discuss.tchncs.de on 03 Jul 09:56 collapse

no, that would not be a problem as soon as the other image library has an api reitti could query. It just happens that I am settled with immich and had no other needs at the moment.

If you need a specific one, drop a feature request and I will have a look.

Zagorath@aussie.zone on 03 Jul 10:09 collapse

I don’t actually have any personally. I’m still with Google Photos for now and hadn’t decided what to switch to, with Immich, Nextcloud, and the non-open Synology Photos being the top of my list. Legitimately, what a tool like this supports could be a factor I use to help decide.

How complicated is the code interfacing with Immich? Is it a piece someone not familiar with your overall code base could relatively easily pick up and make a pull request for?

danielgraf@discuss.tchncs.de on 03 Jul 10:28 collapse

I do not think it is that complicated. The front-end sends a request to the back-end with the current selected day. This triggers a search in Immich returning all photos taken on that specific day. This is returned to the front-end and this than does the heavy lifting like filtering them to the current map bounds, displaying them on the map at a specific location. We proxy all request from the front-end through our server because of CORS issues and I did tried to avoid having to configure Immich besides creating a token for the API.

One would need to either create a specific IntegrationService like ImmichIntegrationService and then figure out a way how the user can configure that. The easiest would be that we just then call all available ones even if I do not see the use case of having multiple Photo-Servers. But it would make the code in Reitti cleaner and would not hurt if we do not configure 20 simultaneous servers :D

Kernal64@sh.itjust.works on 03 Jul 10:22 next collapse

This looks really cool! I just have one question. Is it possible to just install this like normal software on a Linux machine or does it require Docker?

danielgraf@discuss.tchncs.de on 03 Jul 10:34 collapse

Thank you.

At the moment i do not have any plans of providing a way of running it without docker. Mainly because of time to support that.

Since it is a Spring-Boot-Application it would be possible to create a jar file which you can execute or deploy as a service with systemd. But then you have to make sure all prerequisites are also running. That is the one thing I like about docker and especially docker compose.

But short answer: Yes, it is possible but you are on your own at the moment. I would help and maybe we can add a section to the readme how to do it.

kameecoding@lemmy.world on 03 Jul 13:54 collapse

Not sure if you know about it and I haven’t used it myself yet, but being able to create native executables could be relatively easy with this

docs.spring.io/spring-boot/…/index.html

danielgraf@discuss.tchncs.de on 03 Jul 14:06 collapse

I used that once on a past gig and it wasn’t very pleasant to use. Especially in combination with spring boot. But that is a couple of years ago. Maybe things have changed. I personally would prefer the executable jar from spring boot. With that you do not have to make that many steps to make it work. But thanks for the suggestion :)

SomeDudeFromSpace@lemmy.ml on 03 Jul 10:35 next collapse

Great project! Any recommendations for an iOS app for this? I’ve been using OwnTracks, but it works very unreliably. As far as I understand, it’s an OS problem since Apple allows for very limited background processing for the sake of saving battery, so there’s not much any logging apps can do, but I was just curious if someone found a way around it.

danielgraf@discuss.tchncs.de on 03 Jul 11:05 collapse

Not really, I stopped using IOS a year ago because of exactly this reason. Had a lot of problems syncing files because of the power saving. I understand why IOS is doing it and for a normal user I think it is the way to go. But anything beyond that, it only hinders the experience you get out of apps. Maybe someone here as any experience with an app which works reliably.

emrsmsrli@lemmy.world on 03 Jul 10:41 next collapse

What’s the difference from Dawarich, if I may ask? Beside from a better name :)

danielgraf@discuss.tchncs.de on 03 Jul 11:01 collapse

Thanks :) As a German I really like the name Dawarich. First it sound really nice for me but also that “Da war ich” means “There have i been” in german makes, at least for me, an awesome project name.

Take this with a grain of salt because I have no idea what the plans are for Dawarich or have ever been and this is solely based on my external view. For me the main differences are:

  • visits and trips are our main data, everything else is just the way to calculate them. For Dawarich it looks to me, that it is the other way around. It displays all the location data in good way with the heatmap and so on but visits or places seems so tacked on. This should not be an offense against it. I actually still have an instance running and it was the main pushing point to finally start working on Reitti.
  • the sleek UI but this depends on your taste

In the end, they are not that far off. Maybe a matter of taste.

emrsmsrli@lemmy.world on 03 Jul 14:20 next collapse

Thanks for the reply! I will give it a go :)

The word dawarich does not produce a polite sound in my main language, meant no offense to the project :)

pipariturbiini@sopuli.xyz on 03 Jul 21:24 collapse

Maybe slightly off-topic, but how did you end up with the name “reitti”? You say you’re German - do you have some sort of tie to the Finnish language?

danielgraf@discuss.tchncs.de on 04 Jul 02:54 collapse

Oh, i had the idea in mind what i want to create and than it was a matter of a couple of Google queries but in the end one of the LLM suggested a list of different names in foreign languages and reitti somehow sticked 😊

warmaster@lemmy.world on 03 Jul 11:35 next collapse

This looks amazing, congratulations and thank you for making it FOSS. I was wondering if you are considering integrating with Home Assistant.

danielgraf@discuss.tchncs.de on 03 Jul 11:41 collapse

Thanks :)

No, did not occur to me. What would the integration look like? Connecting it to the message bus to receive location updates? Honestly it is a couple years ago I played with HA.

rumba@lemmy.zip on 03 Jul 12:00 next collapse

Location sensor would be a good minimum bar.

A custom card for your app that is just basically a iframe into your app with auth would also be pretty decent. Your version of a map looks really nice.

Maybe surfacing metrics of distance traveled or number of geolocations.

I’ll have to install the app and play around with it to make other recommendations but those are the first things that come to mind.

pyr0ball@lemmy.dbzer0.com on 03 Jul 14:41 next collapse

Home automation using geofencing, and my partner likes to get a notification when I’m heading home from the office

warmaster@lemmy.world on 03 Jul 22:33 collapse

I have the HA app on my phone, it reports my location back to my HA server.

I would like if Reitti could retrieve my location from my HA server, instead of asking me to upload it again to Reitti. Uploading my location in short intervals drains the battery very fast, it’s something I want to avoid if I can.

Additiinally, I don’t want to expose anything to the internet. So I pay for the Home Assistant cloud subscription that does it for me in a more secure manner than what I could implement with the little free time I have. Reitti could retrieve my location more securely if it did so without exiting my LAN.

danielgraf@discuss.tchncs.de on 04 Jul 02:51 collapse

I had a similar setup with Home Assistant in the past so I understand your usecase. For Reitti to detect visits somewhat reliable it needs at least one datapoint of location data a minute. We build location clusters with minimum 5 points in 5 minutes. If HA tracks that often it should work. HA probably tracks more than that.

I could add an integration that Reitti fetches the data from Home Assistant. Do you mind in creating a feature request?

gullmar@feddit.it on 03 Jul 12:08 next collapse

Very very cool! I see that you can import GPX files: can you also do the opposite, i.e., export a GPX file for a selected time range?

danielgraf@discuss.tchncs.de on 03 Jul 12:33 collapse

I was thinking about that, but the main problem is that we do not store all the data which comes in.

If we ingest data from an app, I am pretty sure that the quality of the data is actually usable. But for example if we import an Records.json from Google Takeout. The quality of the earlier years is somewhat sketchy. For this we filter out some points like travelling with over 2000 km/h, sudden direction changes etc and they are lost forever. At least for Reitti they are unknown.

The feature would need a lot of explanation why the data we export is not the same we import.That is the reason I did not implemented it even if it would come in handy for testing stuff. Handling GPX files is a pita …

gullmar@feddit.it on 03 Jul 20:58 collapse

I understand. Currently, I’m using the NextCloud’s extension Phonetrack, and one on my main use cases is to export GPX files to use them in Darktable and add geotracking information to the pictures I shoot with my camera. It works well, actually, but Phonetrack is not too intuitive and does not offer many other features: that’s why I was looking with interest at this alternative!

wise_pancake@lemmy.ca on 03 Jul 13:08 next collapse

Visually this is gorgeous

Really nice work!

illusionist@lemmy.zip on 03 Jul 14:20 next collapse

That looks very neat. Thanks for starting this!

I guess this will directly compete with dawarich, right?

github.com/Freika/dawarich

danielgraf@discuss.tchncs.de on 03 Jul 14:39 collapse

I would not say compete. They are different in how things are done from my point of view. I want to focus more on the visits we have done in the past to relive some lost memories whereas Dwarich looks more “technical” for me. I have no better words for it, I hope you get my point in what i am trying to achieve with Reitti. So there should be enough room for both 🙂

I also do not have any intentions to offer a hosted version in the foreseeable future or even anytime.

illusionist@lemmy.zip on 03 Jul 16:04 collapse

I get what you mean. The focus is different of dawarich. I’m really looking forward into checking reitti out!

TheHobbyist@lemmy.zip on 03 Jul 17:26 next collapse

Congrats! Amazing project, exciting interface and you went the extra mile on the integration side with third parties. Kudos!

Edit: I’ll definitely have to try it out!

umbrella@lemmy.ml on 03 Jul 17:34 next collapse

can i use it to track vehicles too like traccar?

danielgraf@discuss.tchncs.de on 03 Jul 19:56 collapse

I have no experience with traccar but it seems that it supports live tracking. This is something Reitti does not support. Depends on your usecase, but i think traccar is better suited.

SkaveRat@discuss.tchncs.de on 03 Jul 21:28 next collapse

Saving for later

douglasg14b@lemmy.world on 03 Jul 22:05 collapse

Samesies

someacnt@sh.itjust.works on 03 Jul 23:25 next collapse

I am slightly bothered by how the logo is quite off-center, is it intentional?

danielgraf@discuss.tchncs.de on 04 Jul 03:00 collapse

It was not intentional but after bothering not about it because i had other things on my mind i got used to it and now like it the way it is.

But for everyone who is bothered by that. If Reitti reaches 1k stars on Github I will add a switch to use a centered one 😊

ada@piefed.blahaj.zone on 04 Jul 01:55 next collapse

I managed to break our instance. I imported several years worth of google takeout location data, and now the "stay-detection-queue" is stalled.

danielgraf@discuss.tchncs.de on 04 Jul 02:57 collapse

Congratulations 😆

To help with that I would need some information:

  • does it show anything in the logs?
  • what do you mean by several years or how big was the Records.json?

Thank you for testing 🙂

ada@piefed.blahaj.zone on 04 Jul 03:34 collapse

It's a 1gig json file that has about 10 years of data. I get multiple repeats of the rabbit timeout in the logs. The Job Status section tells me that it's got just under 9 hours of processing remaining for just over 16,000 in the stay-detection-queue. The numbers change slightly, so something is happening, but it's been going for over 12 hours now, and the time remaining is slowly going up, not down.

reitti-1 | 2025-07-04T03:06:08.820Z INFO 1 --- [ntContainer#2-1] c.d.r.s.p.VisitDetectionService : Detected 61806 stay points for user ada reitti-1 | 2025-07-04T03:06:17.848Z WARN 1 --- [ntContainer#2-1] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it reitti-1 | reitti-1 | com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 9 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0) reitti-1 | at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.checkShutdown(BlockingQueueConsumer.java:493) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.nextMessage(BlockingQueueConsumer.java:554) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1046) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1021) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1423) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1324) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at java.base/java.lang.Thread.run(Unknown Source) ~[na:na] reitti-1 | Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 9 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0) reitti-1 | at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:528) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:349) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:193) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:125) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:761) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection.access$400(AMQConnection.java:48) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:688) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | ... 1 common frames omitted

danielgraf@discuss.tchncs.de on 04 Jul 04:01 collapse

Thanks for the information. I will try to recreate it locally. In my testing I used a 600MB file and this took maybe 2 hours to process on my server. It is one of these ryzen 7 5825U. Since Reitti tries to do these analysis on multiple cores we start it with 4 to 16 Threads when processing. But the stay detection breaks when doing it that way, so it is locking per user to handle that. If now one of them takes a long time the others will break eventually. They will get resheduled 3 times until rabbitmq gives up.

On what type of system do you run it?

I will add some switches so it is configurable how many threads are opened and add some log statements to print out the duration it took for a single step.

ada@piefed.blahaj.zone on 04 Jul 04:07 collapse

i7-8700 with 64GB of RAM

danielgraf@discuss.tchncs.de on 04 Jul 04:17 collapse

Hmm, I had hoped you say something like a Raspberry PI :D

But this should be enough to have it processed in a reasonable time. What I do not understand in the moment is, that the filesize should not affect it in any way. When importing it 100 Geopoints are bundled, send to RabbitMQ. From there we retrieve them, do some filtering and save them in the database. Then actually nothing happens anymore until the next processing run is triggered.

But this than works with the PostGis DB and not with the file anymore. So the culprit should be there somewhere. I will try to insert some fake data into mine and see how long it takes if i double my location points.

ada@piefed.blahaj.zone on 04 Jul 04:40 collapse

I was also trying to set up GPSLogger whilst it was crunching through the backlog, and I manually transferred a file from that app before I had autologging configured. Not sure if that could have done it?

The times don't overlap, as the takeout file is only up until 2023

danielgraf@discuss.tchncs.de on 04 Jul 04:56 collapse

Thanks for getting back to me. I can look into it. I don’t think it’s connected, but you never know.

The data goes the same way, first to RabbitMQ and then the database. So it shouldn’t matter, it’s just another message or a bunch of them in the queue.

ada@piefed.blahaj.zone on 04 Jul 07:13 collapse

Ok, so it may not be frozen. The numbers in the queue seem to imply it is, however, timelines and places are slowly filling out in my history. A couple of dates I had looked at previously were showing me tracklogs for the day, but not timeline information, and now, they're showing timelines for the day

pfr@lemmy.sdf.org on 04 Jul 03:53 collapse

Hell yes! I turned off location data for immich but now I can use this!

danielgraf@discuss.tchncs.de on 04 Jul 04:20 collapse

It is actually awesome if you have some old photos with the geodata attached and scim through Reitti and suddenly one of them shows up :)