Introducing mdapi
By Pierre-Yves on Thursday, November 19 2015, 15:22 - Général - Permalink
I have recently been working on a new small project, an API to query the information stored in the meta-data present in the RPM repositories (Fedora's and EPEL's).
These meta-data include, package name, summary, description, epoch, version, release but also changelog, the list of all the files in a package. It also includes the dependencies information, the regular Provides, Requires, Obsoletes and Conflicts but also the new ones for soft-dependencies: Recommends, Suggests, Supplements and Enhances.
With this project, we are exposing all this information to everyone, in an easy way.
mdapi will check if the package asked is present in either of the updates-testing, updates or release repositories (in this order) and it will return the information found in the first repo where there is a match (and say so) So for example: https://apps.fedoraproject.org/mdapi/f23/pkg/guake?pretty=True*
shows the package information for guake in Fedora 23, where guake has been updated but the latest version is in updates not updates-testing. Therefore it says "repo": "updates".
The application is written entirely in python3 using aiohttp which is itself based on asyncio, allowing it to handle some load very nicely.
Just to show you, here is the result of a little test performed with the apache benchmark tool:
$ ab -c 100 -n 1000 https://apps.fedoraproject.org/mdapi/f23/pkg/guake This is ApacheBench, Version 2.3 <$Revision: 1663405 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking apps.fedoraproject.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Python/3.4 Server Hostname: apps.fedoraproject.org Server Port: 443 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,4096,128 Document Path: /mdapi/f23/pkg/guake Document Length: 1843 bytes Concurrency Level: 100 Time taken for tests: 41.825 seconds Complete requests: 1000 Failed requests: 0 Total transferred: 2133965 bytes HTML transferred: 1843000 bytes Requests per second: 23.91 [#/sec] (mean) Time per request: 4182.511 [ms] (mean) Time per request: 41.825 [ms] (mean, across all concurrent requests) Transfer rate: 49.83 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 513 610 207.1 547 1898 Processing: 227 3356 623.2 3534 4025 Waiting: 227 3355 623.2 3533 4024 Total: 781 3966 553.2 4085 5377 Percentage of the requests served within a certain time (ms) 50% 4085 66% 4110 75% 4132 80% 4159 90% 4217 95% 4402 98% 4444 99% 4615 100% 5377 (longest request)
Note the:
Time per request: 41.825 [ms] (mean, across all concurrent requests)
We are below 42ms so (0.042 second) to retrieve the info of a package in the updates repo and that's while executing 100 requests at the same time on a server that is in the US while I am in Europe.
- Running at https://apps.fedoraproject.org/mdapi/
- Sources: https://pagure.io/mdapi/
- Note the ?pretty=True in the URL, this is something handy to view the JSON
returned but I advise against using it in your applications as it will increase the amount of data returned and thus slow things down.
Note2: Your mileage may vary when testing mdapi yourself, but it should remain pretty fast!