December 2021 - Announce - AlmaLinux List Archives

Proposal and Request For Feedback: Implement `dnf countme`
by Jonathan Wright 16 Dec '21

16 Dec '21

Hello I am Jonathan Wright, Infrastructure Team Lead for AlmaLinux. I manage most of the plumbing that keeps things humming smoothly along and I’ve been working on some improvements to some parts of it to make things more user friendly for our community. AlmaLinux values transparency <https://wiki.almalinux.org/Transparency.html> and communal decision making, it’s one of the reasons why I decided to become a contributor. As part of some of the work I’m doing I’d like to request some feedback from the community on a proposal to enable `dnf countme` similar to the way the Fedora project does. countme is a core feature of DNF implemented upstream in Fedora 32 (dnf 4.2.9). It is described by the docs as such: Determines whether a special flag should be added to a single, randomly chosen metalink/mirrorlist query each week. This allows the repository owner to estimate the number of systems consuming it, by counting such queries over a week's time, which is much more accurate than just counting unique IP addresses (which is subject to both overcounting and undercounting due to short DHCP leases and NAT, respectively). The flag is a simple "countme=N" parameter appended to the metalink and mirrorlist URL, where N is an integer representing the "longevity" bucket this system belongs to. The following 4 buckets are defined, based on how many full weeks have passed since the beginning of the week when this system was installed: 1 = first week, 2 = first month (2-4 weeks), 3 = six months (5-24 weeks) and 4 = more than six months (> 24 weeks). This information is meant to help distinguish short-lived installs from long-term ones, and to gather other statistics about system lifecycle. countme was designed with privacy in mind and does not add any identifying or unique information to requests so there is no tracking involved. Just a simple “hello” to the repository. Currently, AlmaLinux does not track any sort of usage statistics for our distribution at all. We can technically try to aggregate basic metrics from HTTP logs on our mirrorlist servers but the reliability of the data will not be the best since counting unique IPs is undermined by things like NAT and dynamic addressing. So, I’d like to propose we implement “countme=1” in our repository configs just as Fedora and EPEL have done. I’d also like to propose that the aggregated data be made available publicly, similar to https://data-analysis.fedoraproject.org/ for the community to see. I’ve setup a form for feedback at https://forms.gle/BShXoxJmsjNbMXCk6 in case you’d like to give any input on this proposal. We will keep this form open for about a week. FAQ: Q: When are “countme” requests sent? A: Once a week at random during normal dnf activity. If you do not use dnf calls that would otherwise trigger mirrorlist requests (makecache, install, update) this flag will NOT cause dnf to go out of its way and make special requests. Q: What extra data will be sent that is not currently collected? A: “countme=X” will be added to a random mirrorlist request each week from DNF where X is a number, 1-4 which represents the number of weeks your system has been installed. See above for the explanation of this from the DNF documentation. Q: Will aggregated data be made publicly available? A: Yes Q: What data do you use? A: The only data we look at is in the HTTP request itself. Our log lines are in the standard Combined Log Format. Ex: 172.30.61.81 - - [15/Dec/2021:17:02:12 +0000] "GET /mirrorlist/8/baseos?countme=4 HTTP/1.1" 200 629 "-" "libdnf (AlmaLinux 8.3; generic; Linux.x86_64)" We only look at log lines where the request is "GET", the query string includes "countme=N", the result is 200 or 302, and the User-Agent string matches the libdnf User-Agent header. The only data we use are the timestamp, the query parameters (repo, arch, countme), and the libdnf User-Agent data. In the future we will also aggregate data by country using GeoIP. Our processing and aggregation does not care about IPs themselves or their uniqueness. When we implement the aggregation of geographic data it will use MaxMind’s GeoIP database locally to turn the IP into a region which will be used for tallying generalized metrics for that region. Raw access logs are archived in case we find major issues in any of our processing which would allow us to re-parse the data in the future and correct the published statistics. Q: Can I opt out? A: Yes, but we’d prefer you not since the data is very helpful. The only extra data you’ll be submitting is “countme=X” in one request per week. If you’d like to opt out you can comment out the “countme=1” line in the repository config files in /etc/yum.repos.d/ Discussion for this should be directed to the AlmaLinux Infrastructure mailing list. You can join the list at https://lists.almalinux.org/mailman3/lists/infra.lists.almalinux.org/ -- Jonathan Wright AlmaLinux Foundation Mattermost: chat <https://chat.almalinux.org/almalinux/messages/@jonathan>

1 0

AlmaLinux for Azure HPC --Available Now!
by Jack Aboutboul 14 Dec '21

14 Dec '21

Hello HPC Fans. A few weeks ago we updated our images on Microsoft Azure to 8.5. Today we are proud to announce the general availability of–AlmaLinux OS for Azure HPC <https://azuremarketplace.microsoft.com/en-us/marketplace/apps/almalinux.alm…> based on AlmaLinux OS 8.5. This was a tremendous team effort starting from community requests and developed in collaboration between the AlmaLinux community and the team over at our amazing sponsors Microsoft Azure. We had a couple of goals when we set out on this project. First, is to further enable scientific computing and research. That is something that we feel very strongly about and many across the vast expanse of scientific research have begun to start leveraging the cloud due to the lack of resource constraints there. It is much easier to do you work whether it be simulations or something else if you can add nodes instantly with a quick command in your terminal. Second, HPC is often out of reach for many due to the large investment that is typically required. Combining the benefits of cloud economies-of-scale along with HPC technology opens the door for many to be able to experience what HPC is and what it really do. We hope that it will inspire those already there to explore what more they can accomplish and inspire the next generation of scientists and researchers to begin experimenting. You can now take advantage of Azure’s leading edge H-series CPU and N-series GPU based instances, along with RDMA support and low latency networking to power all of your cloud-based HPC workloads large scale computation, circuit design, fluid dynamic analysis, natural resource exploration and many other HPC workloads. The image includes a suite of the most popular HPC tools and libraries pre-installed including the NVIDIA/Mellanox OFED drivers, InfiniBand-based MPI Libraries such as HPC-X and OpenMPI, Communication runtimes Libfabric and OpenUCX, AMD BLIS, FFTW and FLAME libraries, Intel oneAPI Math Kernel Library, and a host of other domain-specific libraries and utilities. Join us <https://chat.almalinux.org>. Our team is always on the lookout for more contributors. Report any bugs you may see on the Bug Tracker <https://bugs.almalinux.org>. Join the AlmaLinux Community Chat <https://chat.almalinux.org> or our HPC Forum <https://almalinux.discourse.group/c/sigs/hpc/31> if you need any help, post a question, or even if you just want to hang out. Reach us on Reddit <https://reddit.com/r/almalinux> and on Twitter <https://twitter.com/almalinux>. –Happy Hacking

1 0

Welcome Codenotary! AlmaLinux OS Foundation's First Platinum Member
by Jack Aboutboul 09 Dec '21

09 Dec '21

About two months ago we eagerly announced that we were able to then open memberships for the AlmaLinux OS Foundation <https://almalinux.org/blog/what-almalinux-foundation-membership-means-for-y…>. I outlined why I thought it was important that we’d structured the governance of the project this way–in order to give community members true ownership and a direct voice in the direction of AlmaLinux OS. The reception was tremendous and we’ve had hundreds of applicants and we’ve so far approved, I believe over 100 community members and dozens of mirror members. It’s very encouraging and refreshing to see the community rise to the occasion. I salute each and every person who has been active in our wonderful community, has helped to make it such a welcoming, friendly and diverse place and has taken the step to make their voice heard. We also have a membership tier for corporations who would like to participate in governance. We know that thousands (and probably tens of thousands) of groups, organizations and companies relied on the old CentOS Linux for a wide array of reasons and we assumed that they would like more direct participation in whatever solution they chose to move forward with. We’ve spoken to several already and you’ll be hearing more about them shortly but today we are here to celebrate Codenotary <https://codenotary.com/>–our first platinum member! You can read more about their take on the Codenotary Blog <https://codenotary.com/blog/codenotary-joins-almalinux-foundation>. Codenotary is a pioneer in DevSecOps and immutability ( https://github.com/codenotary/immudb) and offer developer friendly tools (check out the Community Attestation Service https://cas.codenotary.com/ amongst others) that make truly verifiable software supply chain security a reality–today. They have a far reaching vision and care deeply about ensuring that the software you run, build and use is software that you (and your users) can trust. Behind Codenotary also stands Moshe Bar, who is an open source visionary and brought Xen and KVM to the world. He’s been a lifelong staunch supporter of the Open Source community and his vote of confidence means the world to us. Codenotary’s support will help us to continue delivering for the community day in and day out, and we’ll be working together to continue to grow the CentOS ecosystem and make sure AlmaLinux is the secure base you can build your future on. This moment is also a further step towards independence for the AlmaLinux OS Foundation. As many people know, CloudLinux <https://cloudlinux.org/> were the ones that kicked off the AlmaLinux OS project and foundation. The intention always was to get it up on it’s feet so that it could be a truly independent body governed and serving the community. Igor Seletskiy’s bold move stepping down from the board <https://blog.cloudlinux.com/why-i-have-decided-to-step-down-from-the-almali…> was the first step towards ensuring that no single entity has an overwhelming amount of control on the project. For the next step towards independence, I will be transitioning out of CloudLinux and into a role at Codenotary and they will be sponsoring my work. I will retain my position on the board too–don’t worry you can’t get shake me that easily. This leaves CloudLinux with just one board seat until elections are held. As for me nothing changes, I’ll continue to make sure that we deliver everything we promise, and more, for the community. That’s why we are here. It’s our reason for doing it every day. It is an honor to serve the community. So it’s not New Year’s yet but don’t mind if we break out the champagne a little early. On behalf of the whole AlmaLinux OS Foundation community and the board, benny, Simon, Jesse, Eugene and myself we would like to thank and toast Moshe Bar and the whole Codenotary team. We are immeasurably thankful for your support.

1 0