How can I effectively search all activities?

[stefcameron]

I have 13 years’ worth of Garmin data that I have now imported into Intervals.icu and most activities have names of climbs or special “tags” in their names (and descriptions, that sadly didn’t get imported) so I can find similar ones to compare them.

Problem is, I can’t seem to search more than about 2 years’ worth of activities at any given time before the UI seems to just give up.

I have a similar issue when I want to see my “all time” fitness chart. If I go on the Fitness tab and select all 13 years, I get the first ~2 most recent years and then it spins forever and never loads more into the chart.

More details if needed…

If I go to the Activities page, switch to list view, and set the range to Aug 20, 2010 (first recorded activity ever) to today, along with a filter on the name, it starts to list activities but stops after going back ~2 years from the first hit, and then the spinner just spins forever. So if the first found activity was in 2019, it’ll list others back to ~2017 and then spin forever, never finding anything else.

With the API and a query like https://intervals.icu/api/v1/athlete/ATHLETE_ID/activities/search?q=FOO, I can get all the results almost instantly, albeit that gets just a tiny bit of data from the server so maybe that’s part of the problem.

With Today’s Plan (I’m coming from there and checking things out here), it was easy to see my all time fitness history/progression, or do a “full text search” on all my activities to find stuff.

[stefcameron]

I actually managed to get my “all time” fitness chart, but in order to generate it, I had to start with the 2 most recent years (2021-2023), and then once it was done, add the previous 2 years (extending the range to 2019-2023), and then another 2, and another 2, until I finally got down to 2010. Took about 15-20 minutes in total to generate it.

I feel like there’s something in the app that falls apart when it has to load more than 2 year’s worth of data in one “request” by the user.

[david]

You can use the search box on the calendar page to quickly find and navigate to activities by name or #tag:

Activity tags can be captured or assigned from a workout when one is paired with an activity.

If you try and load 13 years worth of data on the calendar page, activity list view or fitness page the app will struggle. That’s a huge amount of stuff to fetch from the server and render. All the activities are sent back via web sockets. I think the buffers on the server fill up before the data can be sent to the browser which is maybe why you managed to get it to work by doing a couple of years at a time. I might be able to fix that but the app will be slow with that many activities.

This isn’t something I have tried to optimise for because its not something people do often

You can get your all time power curve on the /power page. That is computed server side so is fast.

Anyway let me know how you get on. I haven’t used Today’s Plan much so I can’t give you too many pointers. One thing I did like of theirs was their power vs HR plot. I built a better one (IMO) in Intervals.icu (see /compare page). I managed to get a little fitter in 2023 vs 2022:

It sounds like you might be a software developer? If you know some Javascript you can extend Intervals.icu in many places:

Extending Intervals.icu

This post pulls together all the ways you can extend Intervals.icu. Some require a little Javascript knowledge and others just picking a value from a drop down. Intervals.icu also has an API.

Also checkout the guides:

Guide

[stefcameron]

Happy New Year!

@david

You can use the search box on the calendar page to quickly find and navigate to activities by name or #tag:

Aha! I never thought to look under “OPTIONS” (in Calendar view only) for the kind of “full search” I’ve been hunting around for, but this appears to basically be calling that /activities/search?q=OLH endpoint and showing the results (unordered name/date only) very quickly over “all time”.

My first instinct was to try this field in Intervals’ left sidenav, but I still haven’t figured out what this is for (and didn’t manage to find an explanation in the Guides; no handy tooltip on hover over the icon either ):

Now if only the results from “OPTIONS > Search” could be ordered… and if I could see some stats for each one, similar to how it looks in the “Activities” page in list view:

If you’re curious, in Today’s Plan, they have this toolbar at the top which has this icon

Clicking it gives you this “Quick file search” modal (it floats above the Calendar and takes up about 95% of the browser window) where you can filter by activities, any date range you want, and some text to search activity names including notes/descriptions (something which Intervals doesn’t search, AFAICT):

From there, you can click an activity to open its details in a modal so you can come back to your search, and it’s easy to compare a few things just by looking. (Within an activity, you can load in another activity to compare in Analysis view.)

Since Intervals doesn’t appear to search what you call a Description associated with an activity, I’m going to use the API to reformat all my activity Names to include climb/hill names.

In Garmin, those are all in activity descriptions because Garmin activity titles/names are limited to 68 characters (and the activity import of Garmin history into Intervals didn’t include descriptions, so those are all missing in Intervals anyway).

But at least in Garmin, an “all activities” search includes both titles and descriptions so I can still find everything over there (though I rarely use Garmin Connect for anything other than building/syncing routes and setting title/desc in case I ever have to sync history elsewhere, like I just had to do a few days ago with Intervals).

@david

One thing I did like of theirs was their power vs HR plot. I built a better one (IMO) in Intervals.icu (see /compare page).

That is a cool chart! I like your version of it. Easy to interpret!

@david

It sounds like you might be a software developer? If you know some Javascript you can extend Intervals.icu in many places:

Yes, sir! I’m amazed at what you’re managed to build here, not to mention all the extensibility you’ve added to it via the custom fields with JS and the API. I’ve already customized my activity view to give me fields I previously had in Today’s Plan (and more) which I found so handy there:

[MedTechCD]

The first search screenshot you posted, searches other athletes which you’re following/coaching.

You can set plenty of filters in the activity list view. Name/ Not Name/description/tags…
Isn’t that together with the date range an alternative to your search question?

[stefcameron]

@MedTechCD

The first search screenshot you posted, searches other athletes which you’re following/coaching.

Got it, thank you!

@MedTechCD

You can set plenty of filters in the activity list view. Name/ Not Name/description/tags…
Isn’t that together with the date range an alternative to your search question?

It would be if only it didn’t suffer from the ~2 year max data pull issue (2 years with my data; might be more/less with someone else’s), which I assume is related to the use of web sockets to pull the data from the server, which Dave mentioned earlier.

The Activities List View was my first go to. The first thing I did was set my date range from Aug 20, 2010, to Dec 31, 2023, and then the app fell over. I did also set a name filter to find rides by my coding system, but since those go back further than 2021, when the app had loaded data from Dec 31, 2023, back to around early 2021, it just stopped. The spinner in the top/right corner of the list view kept spinning indefinitely and nothing more ever loaded.

So the OPTIONS > Search feature (Calendar View only) seems to be the only way, currently, to do a quick name-based search across “all history”.

I just wish it would show results in a way I could sort and filter just like I can with a relatively small subset of data in the Activity List view, and that it included descriptions, not just names – if at least with pre-computed data found in FIT files like total ascent, moving time, distance, speed, load, etc (not necessarily activity streams or things that require further data crunching).

[MedTechCD]

I see now…
I only quickly scanned your former post without fully understanding.
Sounds like a challenge…

[Gerald]

How many activities do you have?
I start from 2017-2023, then once it’s refreshed I add 2012-2023 and then finally 2008-2023.
2879 activities 2008 - 2024.

Once it has pulled all the data, the second time I open it, it’s much faster.
The searching by name is very quick.

Running Chrome on Win11 (i7, 8GB ram). WKO has a similar problem, with large amounts of data. They advise to let it update all data, then run the various reports.

[stefcameron]

@Gerald

How many activities do you have?
I start from 2017-2023, then once it’s refreshed I add 2012-2023 and then finally 2008-2023.
2879 activities 2008 - 2024.

Once it has pulled all the data, the second time I open it, it’s much faster.
The searching by name is very quick.

I have about 4600 activities from Aug 2010 to today. The app tops out at around 1255 activities, AFAICT, in the Activities List View.

[stefcameron]

FYI, I decided to turn this into a feature request: Improved all activities search

[Gerald]

@stefcameron can you give this a try, and see if your activities load quicker.

1. Create tabs for different activities types (see my example).
   Tabs on the activity list view - #14 by Gerald

2. My RACING tab is relatively short, compared to all ride, runs, all activities, so loaded all races as quickly as it takes to change start and end dates (<10 seconds).
   

   

3. I then clicked on the ALL tab, and it took another +/-10 seconds to load all 2672 activities. This is significantly faster compared to opening the ALL tab first.

[stefcameron]

@Gerald Thank you for thinking about this and for the suggestion! I gave it a try, but it makes no difference. I never got past loading the runs, and that never finished either. It took 5 minutes just to load 67 runs (and I already had about 15-20 of those loaded) dating back to July 2022.

There are hundreds, likely close to 2000 (I have ~4600 activities in total, more rides than runs).

It’s just not workable.

I think what’s needed is a dedicated Search View that uses the fast API I mentioned that returns results based on pre-calculated data already stored on the backend (since I think there’s a lot of data crunching that happens on the frontend that would slow things down significantly with the number of activities to load). I believe this is the API used in “Calendar View > Options > Search for activity” because that is super fast and searches everything through all time (this search took <1s):

But it doesn’t show results in a workable table with at least activity time, distance, total ascent, etc., and it doesn’t search descriptions, only names.

You can see in that list, a search for Old La Honda (famous climb in the SF Bay Area) yields results back to 2012/2013. I never find those in the List View because they never load (unless I really persist at it by gradually increasing the date range, but it takes 15-20 minutes).

And the search should also include descriptions so I don’t have to stuff everything into the Name and end-up with super long names like this (because I like to search on hill names; e.g. what are all the times I did this climb?):

[Gerald]

Have you tried exporting to Excel, using the API?
I know this defeats the point of using one application, but it’s as simple as click on “refresh data”.

APPS in EXCEL - a guide to getting started

This section will help provide help when using some useful tools in Excel. The latest version of each file will be linked here too. Current features available RHR / HRV chart Annual Training Plan Calculator - Sweat Rate There are three version available, due to limitations in older versions of Excel, or due to some features not available, eg. MacOS versions. These should be downloaded to you computer, and used offline: [Office 365] <— updated version coming soon [MacOS] ← updated versio…

MacOS is limited in its use of Excel, but Windows versions work seamlessly.

[stefcameron]

@Gerald

Have you tried exporting to Excel, using the API?
I know this defeats the point of using one application, but it’s as simple as click on “refresh data”.

No, I haven’t, and that would be super inconvenient for regular use. I’m not without solutions to this problem, but for a paid app (one I choose to pay for/support), I see activity search (regardless of date) as a core feature.

For now, if I have to find something, I can go to Strava or Garmin Connect since they have search functions that are fast and can search everything (and thankfully, I’ve been diligent to keep those updated over the years, exactly for this reason I’m facing now).

The downside to these other “solutions” is I can’t (without a lot of time on my hands) easily load up the activities I find there into Intervals.icu because I don’t know the activity IDs. If I knew the IDs, then I could at least copy/paste them into the URL and load them directly (but I don’t want to have to export to CSV every time I want to know an activity ID either; I want my cake, and I want to eat it too! ). And then even if I could do that, I wouldn’t be able to compare them.

It’s just a hard sell for me, coming from Today’s Plan, as I detailed in my linked feature request (see one of my previous comments).

But that’s not to take away from everything else that’s really awesome in Intervals.icu. Overall, and for many reasons, it’s a fantastic app and I’m glad to have found it!

[Wazc23]

Hi @stefcameron, thanks for your info above, I actually have seen this also with over 10 years of data. I have created myself three Seasons for 4 year chunks which seems to be working well for me . i.e.

Settings > Seasons

Then on the Activities List page I select the Season and then do my search

[stefcameron]

@Wazc23

Hi @stefcameron, thanks for your info above, I actually have seen this also with over 10 years of data. I have created myself three Seasons for 4 year chunks which seems to be working well for me . i.e.

Thank you for the suggestion, @Wazc23 Seasons are related to lots of measurement/comparison features in Intervals so even if this worked, it would mess up the seasons I already have configured (I don’t think Intervals allows for overlapping seasons either, so I couldn’t define seasons like these that overlap the seasons I already have). And even if this did succeed in loading activities faster for some reason, it still isn’t a solution to activity search because I still can’t search everything. I’m limited to searching within batches.

[david]

I am busy adding a text search to the activity list view. This replaces the date selector when enabled. Will take a bit of time to get this done.

[stefcameron]

Sounds great! Thanks for taking it on!

[david]

It’s live now. I will do an announcement and provide more details once any initial bugs are found.

[david]

Search all activities from list view

The activity list view now supports searching all activities for text. The name and description are searched (case insensitive). If the search text starts with a ‘#’ then the tags are also checked. Click the search button next to the date range selector: [Screenshot 2024-01-22 at 21.38.27] And enter text or a tag to search. You need to type at least 3 characters and up to 100 activities are returned. [Screenshot 2024-01-22 at 21.40.00] Note that the activity filters for the tab still appl…