Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/lib/kkuleomi
Commit message (Collapse)AuthorAgeFilesLines
* Don't recreate the ES-index by defaultMax Magorsch2020-02-121-1/+1
| | | | | | | | | | | By default the ES index will only be created now if it is not alredy present. In case it is already present it won't be recreated when using the kkuleomi:index:init task. To forcefully recreate the index, kkuleomi:index:recreate can be used. Signed-off-by: Max Magorsch <arzano@gentoo.org>
* Remove unused base_settings variableHans de Graaff2019-12-201-21/+0
| | | | Signed-off-by: Hans de Graaff <graaff@gentoo.org>
* rubocop: autofixRobin H. Johnson2019-12-191-2/+1
| | | | Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* Add an ES index containing the git commitsMax Magorsch2019-09-061-0/+1
| | | | | | | | | | | | | | | | | So far, any history-related information has been fetched using git. That is, whenever a user requested any history-related information 'git log' was run and the output was parsed. That is time-consuming. Loading the page https://packages.gentoo.org/packages/keyworded takes around 120 seconds this way. Instead of doing so, any git commits are now added to an ES index and retrieved using ES. This way, the same page as mentioned before, loads in under 3 seconds. The commits for populating the index are fetched incrementally. This way, the first run may take some time, but afterwards, updates are fast. Signed-off-by: Max Magorsch <max@magorsch.de>
* Migrate to ES 7.3 and the repository patternMax Magorsch2019-09-055-269/+23
| | | | | | | | | | | | | | | | | | Elasticsearch-persistence is used as the persistence layer for Ruby domain objects in Elasticsearch in this application. So far, the ActiveRecord pattern has been used here. However, this pattern has been deprecated as of version 6 of the gem and was removed in version 7. That's why the application has been migrated to use the repository pattern instead. For further information, please see: https://www.elastic.co/blog/activerecord-to-repository-changing- persistence-patterns-with-the-elasticsearch-rails-gem Note: The old Elasticsearch index won't be compatible with this version anymore. That's why a fresh index should be populated. Signed-off-by: Max Magorsch <max@magorsch.de>
* False instead of strict, strict is for types.dynamic2Alec Warner2018-08-251-1/+1
|
* Turn off dynamic field detection.dynamicAlec Warner2018-08-251-1/+2
|
* Fix broken deletion of versions in import code.v6.0.6Alec Warner2018-02-261-1/+1
| | | | | | The previous version relied on elastic search matching packages based solely on package name (e.g. "gentoo-sources") but in order to collect the version from the index, an entire atom is required.
* kkuleomi/store: bump field limit.v6.0.3Robin H. Johnson2018-02-211-1/+1
| | | | Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* Push the size param into the aggregation part.v6.0.2Alec Warner2018-02-181-3/+6
| | | | | | | | | | According to https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html this controls the number of buckets (defaulting to 10 apparently). This was changed in ES5 but went unnoticed until now. Force to 10k buckets to continue to support flags with *lots* of packages and revisit in the future if too expensive.
* packages_search: raise result limit for USE-flag searchRobin H. Johnson2018-02-181-0/+1
| | | | | | | Some USE flags like static-libs are very common, so raise the limit. Bug: https://bugs.gentoo.org/648040 Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* Fix Changes index again.Alec Warner2018-02-171-1/+1
| | | | | Its simpler to fix by not renaming the class and just renaming the index name due to how we automate things in p.g.o.
* Fix index typo.Alec Warner2018-02-171-1/+1
|
* Fix bug where field limits were not being raised.Alec Warner2018-02-041-6/+4
| | | | Merge base_settings into settings used to create index.
* Fix find_all_by_package.Alec Warner2018-02-042-14/+1
| | | | | | Originally designed as a replacement for find_all_by_parent, we don't need a custom function for it because Version.find_all_by(:package, name) works just as well.
* order-by _term is deprecated, replaced with _key.Alec Warner2018-01-201-1/+1
| | | | https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html
* Remove size from sub-query.Alec Warner2018-01-201-1/+0
| | | | | size: 0 goes out outside of the query (and already exists in this query); dropping it fixes queries.
* Turn category names back into keywords.Alec Warner2018-01-201-1/+1
| | | | | | | The 'text' fields were not indexed properly. Keyword fields are indexed properly, so turn it back into a keyword. Also fix the index reloading to remove indices by name.
* Boost factor is replaced by weight.Alec Warner2018-01-191-1/+1
| | | | https://www.elastic.co/guide/en/elasticsearch/reference/1.4/query-dsl-function-score-query.html
* Fix gateway. Allow more fields in indices.Alec Warner2018-01-171-2/+8
| | | | | Indexes have field limits now. I'm not sure this is the right fix, but I've expanded it to 25k; hoping to contain the entire tree.
* Another parent relation needed to be removed.Alec Warner2018-01-171-1/+1
|
* Drop parent / child relation in version.Alec Warner2018-01-172-2/+1
| | | | | ES6's multiple indexes (as opposed to a single index) mean we cannot use this relation. Instead we key versions off of package atoms.
* Fix extra } causing compilation to fail.Alec Warner2018-01-171-1/+1
|
* Add find_all_by_package.Alec Warner2018-01-172-1/+15
| | | | | | | | | | | Due to splitting of indexes for ES6, we can no longer rely on parent, child relations to determine 'which packages have which versions' because ES6 cannot join between two indexes. Instead we lookup in the versions table a particular package (CP). This should yield some reasonable count of versions (typically 1-100). We then use those results to find answers like "highest version" by simplying sorting the result and taking the first of the sort.
* has_parent does not support filter.Alec Warner2018-01-161-2/+2
| | | | | | ES6 replaces this with 'query'. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-parent-query.html
* Fix Typoes.Alec Warner2018-01-161-5/+5
| | | | Comma separate the list items, of course.
* Update ES indexing scheme for ES6.Alec Warner2018-01-161-26/+15
| | | | | | | | | | | | | | | ES6 introduces a number of breaking changes into ES. 1) https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html This basically entails breaking up the single multi-typel index we had into N indices, one per model. Our other option is to continue to use a single index, but add a custom type property. This seemed unwieldy. 2) The 'String' type was also deprecated. See https://www.elastic.co/blog/strings-are-dead-long-live-strings So we see many updates from "not analyzed" to "keyword" to retain the previous behavior
* Fix typo that broke compilation.Alec Warner2018-01-151-1/+1
|
* Update elasticsearch queries.Alec Warner2018-01-151-6/+4
| | | | | | | Similar to the previous commit: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-filtered-query.html documents that query => filtered semantic is deprecated. This fixes more references.
* "Filtered / Query" is deprecated in ES 5.1Alec Warner2018-01-141-2/+2
| | | | | | https://www.elastic.co/guide/en/elasticsearch/reference/5.1/query-dsl-filtered-query.html "Filtered / Filter" is still valid.
* Manually expire changelog cacheAlex Legler2016-08-102-0/+3
| | | | This should hopefully relieve the load the repeated `git log' calls cause.
* Initial commit w/currently running codeAlex Legler2016-07-209-0/+657