As you may have seen, I sent the following Tweet: “The Apple ARM MacBook future is coming, maybe sooner than people expect” https://twitter.com/choco_bit/status/1266200305009676289?s=20
Today, I would like to further elaborate on that. tl;dr Apple will be moving to Arm based macs in what I believe are 4 stages, starting around 2015 and ending around 2023-2025: Release of T1 chip Macbooks, release of T2 chip Macbooks, Release of at least one lower end model Arm Macbook, and transitioning full lineup to Arm. Reasons for each are below.
Apple is very likely going to switch to switch their CPU platform to their in-house silicon designs with an ARM architecture. This understanding is a fairly common amongst various Apple insiders. Here is my personal take on how this switch will happen and be presented to the consumer.
The first question would likely be “Why would Apple do this again?”. Throughout their history, Apple has already made two other storied CPU architecture switches - first from the Motorola 68k to PowerPC in the early 90s, then from PowerPC to Intel in the mid 2000s. Why make yet another? Here are the leading reasons:
- Intel has, in recent years, been making significant losses both in reputation and in actual product value, as well as velocity of product development, breaking their bi-yearly “Tick Tock” cycle for the first time in decades. Most recently, they have fallen well behind AMD’s processor lines in cost to performance ratio, CPU core count, core design (monolithic design vs “chiplet”), power consumption to performance, silicon supply (Intel with significant manufacturing process and yield issues), and on-silicon security features. While Intel still wins out in certain enterprise and datacenter applications, as well as having a much better reputation for reliability and QA (AMD having shipped numerous chips with a broken random- number generator that prevented even booting some mainstream operating system), the number of such applications slowly dwindles with each new release from AMD, and as confidence among decisionmakers in enterprise increases. In the public consciousness, Intel is quickly becoming a point of ridicule against Apple’s Mac lineup, rather than a badge of honor.
- By moving to their own designs, Apple will be free from Intel’s release schedule, which have recently been unpredictable and faced with routine delays due to poor manufacturing yields. Apple will be able to update their Mac lineup on their own timeline, rather than being forced to delay products based on Intel’s ability to meet the release window. This also allows them to leverage relationships with other silicon fabricators to source chips, rather than relying on Intel ’s continued “iteration” that’s leading to a “14nm++++++++++” process, or the continued lack of product diversity with the 10nm process. Apple will also be free to innovate in the design of the silicon platform, rather than being limited by Intel’s design choices. By having full control of the manufacturing and development cycle, Apple can bring even more in-house optimization to the macOS, as they have been for iOS and iPadOS over the years.
- Using an ARM architecture on the Macs allows for a more unified Apple ecosystem, rather than having separate Mac and iOS-based products. The only distinction will be the device form factor and performance characteristics.
- The x86_64 architecture is very old and inefficient, using older methodologies for processor design (CISC vs ARM’s RISC), and the instruction set continues to require support in silicon for emulating 1980s-vintage 16-bit modes, as well as ineffectual and archaic memory addressing modes (segmentation, etc.) The x86_64 architecture is like a city, built atop a much older city, built atop a yet older city, but every layer is built with NYC infrastructure levels of complexity that suited its time and no further.
- Over the last 10 years, Apple has shown that they can consistently produce impressive silicon designs, often leading the market in performance and capability, and Apple has been aggressively acquiring silicon design talent.
A common refrain heard on the Internet is the suggestion that Apple should switch to using CPUs made by AMD, and while this has been considered internally, it will most likely not be chosen as the path forward, even for their megalithic giants like the Mac Pro. Even though AMD would mitigate Intel’s current set of problems, it does nothing to help the issue of the x86_64 architecture’s problems and inefficiencies, on top of jumping to a platform that doesn’t have a decade of proven support behind it. Why spend a lot of effort re-designing and re- optimizing for AMD’s platform when you can just put that effort into your own, and continue the vertical integration Apple is well-known for?
I believe that the internal development for the ARM transition started around 2015/2016 and is considered to be happening in 4 distinct stages. These are not all information from Apple insiders; some of these these are my own interpretation based off of information gathered from supply-chain sources, examination of MacBook schematics, and other indicators from Apple.
Stage1 (from 2014/2015 to 2017):
The rollout of computers with Apple’s T1 chip as a coprocessor. This chip is very similar to Apple’s T8002 chip design, which was used for the Apple Watch Series 1 and Series 2. The T1 is primarily present on the first TouchID enabled Macs, 2016 and 2017 model year MacBook Pros.
Considering the amount of time required to design and validate a processor, this stage most likely started around 2014 or 2015, with early experimentation to see whether an entirely new chip design would be required, or if would be sufficient to repurpose something in the existing lineup. As we can see, the general purpose ARM processors aren’t a one- trick pony.
To get a sense of the decision making at the time, let’s look back a bit. The year is 2016, and we're witnessing the beginning of stagnation of Intel processor lineup. There is not a lot to look forward to other than another “+” being added to the 14nm fabrication process. The MacBook Pro has used the same design for many years now, and its age is starting to show. Moving to AMD is still very questionable, as they’ve historically not been able to match Intel’s performance or functionality, especially at the high end, and since the “Ryzen” lineup is still unreleased, there is absolutely no benchmarks or other data to show they are worth consideration, and AMD’s most recent line of “Bulldozer” processors were very poorly received. Now is probably as good a time as any to begin experimenting with the in-house ARM designs, but it’s not time to dive into the deep end yet, our chips are not nearly mature enough to compete, and it’s not yet certain how long Intel will be stuck in the mud. As well, it is widely understood that Apple and Intel have an exclusivity contract in exchange for advantageous pricing. Any transition would take considerable time and effort, and since there are no current viable alternative to Intel, the in-house chips will need to advance further, and breaching a contract with Intel is too great a risk. So it makes sense to start with small deployments, to extend the timeline, stretch out to the end of the contract, and eventually release a real banger of a Mac.
Thus, the 2016 Touch Bar MacBooks were born, alongside the T1 chip mentioned earlier. There are good reasons for abandoning the piece of hardware previously used for a similar purpose, the SMC or System Management Controller. I suspect that the biggest reason was to allow early analysis of the challenges that would be faced migrating Mac built- in peripherals and IO to an ARM-based controller, as well as exploring the manufacturing, power, and performance results of using the chips across a broad deployment, and analyzing any early failure data, then using this to patch any issues, enhance processes, and inform future designs looking towards the 2nd stage.
The former SMC duties now moved to T1 includes things like
- Fan speed, voltage, amperage and thermal sensor feedback data
- FaceTime camera and microphone IO
- PMIC (Power Management Controller)
- Direct communication to NAND (solid state storage)
- Direct communication with the Touch Bar
- Secure Enclave for TouchID
The T1 chip also communicates with a number of other controllers to manage a MacBook’s behavior. Even though it’s not a very powerful CPU by modern standards, it’s already responsible for a large chunk of the machine’s operation. Moving control of these peripherals to the T1 chip also brought about the creation of the fabled BridgeOS software, a shrunken-down watchOS-based system that operates fully independently of macOS and the primary Intel processor.
BridgeOS is the first step for Apple’s engineering teams to begin migrating underlying systems and services to integrate with the ARM processor via BridgeOS, and it allowed internal teams to more easily and safely develop and issue firmware updates. Since BridgeOS is based on a standard and now well-known system, it means that they can leverage existing engineering expertise to flesh out the T1’s development, rather than relying on the more arcane and specialized SMC system, which operates completely differently and requires highly specific knowledge to work with. It also allows reuse of the same fabrication pipeline used for Apple Watch processors, and eliminated the need to have yet another IC design for the SMC, coming from a separate source, to save a bit on cost.
Also during this time, on the software side, “Project Marzipan”, today Catalyst, came into existence. We'll get to this shortly.
For the most part, this Stage 1 went without any major issues. There were a few firmware problems at first during the product launch, but they were quickly solved with software updates. Now that engineering teams have had experience building for, manufacturing, and shipping the T1 systems, Stage 2 would begin.
Stage 2 encompasses the rollout of Macs with the T2 coprocessor, replacing the T1. This includes a much wider lineup, including MacBook Pro with Touch Bar, starting with 2018 models, MacBook Air starting with 2018 models, the iMac Pro, the 2019 Mac Pro, as well as Mac Mini starting in 2018.
With this iteration, the more powerful T8012 processor design was used, which is a further revision of the T8010 design that powers the A10 series processors used in the iPhone 7. This change provided a significant increase in computational ability and brought about the integration of even more devices into T2. In addition to the T1’s existing responsibilities, T2 now controls:
- Full audio subsystem
- Secure Enclave for internal NAND storage and encryption/decryption offload
- Management of the whole system’s power and startup sequence, allowing for trusted boot (ensure boot chain-of-trust with no malicious code/rootkit/bootkit)
Those last 2 points are crucial for Stage 2. Under this new paradigm, the vast majority of the Mac is now under the control of an in-house ARM processor. Stage 2 also brings iPhone-grade hardware security to the Mac. These T2 models also incorporated a supported DFU (Device Firmware Update, more commonly “recovery mode”), which acts similarly to the iPhone DFU mode and allows restoration of the BridgeOS firmware in the event of corruption (most commonly due to user-triggered power interruption during flashing).
Putting more responsibility onto the T2 again allows for Apple’s engineering teams to do more early failure analysis on hardware and software, monitor stability of these machines, experiment further with large-scale production and deployment of this ARM platform, as well as continue to enhance the silicon for Stage 3.
A few new user-visible features were added as well in this stage, such as support for the passive “Hey Siri” trigger, and offloading image and video transcoding to the T2 chip, which frees up the main Intel processor for other applications. BridgeOS was bumped to 2.0 to support all of these changes and the new chip.
On the macOS software side, what was internally known as Project Marzipan was first demonstrated to the public. Though it was originally discovered around 2017, and most likely began development and testing within later parts of Stage 1, its effects could be seen in 2018 with the release of iPhone apps, now running on the Mac using the iOS SDKs: Voice Recorder, Apple News, Home, Stocks, and more, with an official announcement and public release at WWDC in 2019. Catalyst would come to be the name of Marzipan used publicly. This SDK release allows app developers to easily port iOS apps to run on macOS, with minimal or no code changes, and without needing to develop separate versions for each. The end goal is to allow developers to submit a single version of an app, and allow it to work seamlessly on all Apple platforms, from Watch to Mac. At present, iOS and iPadOS apps are compiled for the full gamut of ARM instruction sets used on those devices, while macOS apps are compiled for x86_64. The logical next step is to cross this bridge, and unify the instruction sets.
With this T2 release, the new products using it have not been quite as well received as with the T1. Many users have noticed how this change contributes further towards machines with limited to no repair options outside of Apple’s repair organization, as well as some general issues with bugs in the T2.
Products with the T2 also no longer have the “Lifeboat” connector, which was previously present on 2016 and 2017 model Touch Bar MacBook Pro. This connector allowed a certified technician to plug in a device called a CDM Tool (Customer Data Migration Tool) to recover data off of a machine that was not functional. The removal of this connector limits the options for data recovery in the event of a problem, and Apple has never offered any data recovery service, meaning that a irreparable failure of the T2 chip or the primary board would result in complete data loss, in part due to the strong encryption provided by the T2 chip (even if the data got off, the encryption keys were lost with the T2 chip). The T2 also brought about the linkage of component serial numbers of certain internal components, such as the solid state storage, display, and trackpad, among other components. In fact, many other controllers on the logic board are now also paired to the T2, such as the WiFi and Bluetooth controller, the PMIC (Power Management Controller), and several other components. This is the exact same system used on newer iPhone models and is quite familiar to technicians who repair iPhone logic boards. While these changes are fantastic for device security and corporate and enterprise users, allowing for a very high degree of assurance that devices will refuse to boot if tampered with in any way - even from storied supply chain attacks, or other malfeasance that can be done with physical access to a machine - it has created difficulty with consumers who more often lack the expertise or awareness to keep critical data backed up, as well as the funds to perform the necessary repairs from authorized repair providers. Other issues reported that are suspected to be related to T2 are audio “cracking” or distortion on the internal speakers, and the BridgeOS becoming corrupt following a firmware update resulting in a machine that can’t boot.
I believe these hiccups will be properly addressed once macOS is fully integrated with the ARM platform. This stage of the Mac is more like a chimera of an iPhone and an Intel based computer. Technically, it does have all of the parts of an iPhone present within it, cellular radio aside, and I suspect this fusion is why these issues exist.
Recently, security researchers discovered an underlying security problem present within the Boot ROM code of the T1 and T2 chip. Due to being the same fundamental platform as earlier Apple Watch and iPhone processors, they are vulnerable to the “checkm8” exploit (CVE-2019-8900). Because of how these chips operate in a Mac, firmware modifications caused by use of the exploit will persist through OS reinstallation and machine restarts. Both the T1 and T2 chips are always on and running, though potentially in a heavily reduced power usage state, meaning the only way to clean an exploited machine is to reflash the chip, triggering a restart, or to fully exhaust or physically disconnect the battery to flush its memory. Fortunately, this exploit cannot be done remotely and requires physical access to the Mac for an extended duration, as well as a second Mac to perform the change, so the majority of users are relatively safe. As well, with a very limited execution environment and access to the primary system only through a “mailbox” protocol, the utility of exploiting these chips is extremely limited. At present, there is no known malware that has used this exploit. The proper fix will come with the next hardware revision, and is considered a low priority due to the lack of practical usage of running malicious code on the coprocessor.
At the time of writing, all current Apple computers have a T2 chip present, with the exception of the 2019 iMac lineup. This will change very soon with the expected release of the 2020 iMac lineup at WWDC, which will incorporate a T2 coprocessor as well.
Note: from here on, this turns entirely into speculation based on info gathered from a variety of disparate sources.
Right now, we are in the final steps of Stage 2. There are strong signs that an a MacBook (12”) with an ARM main processor will be announced this year at WWDC (“One more thing...”), at a Fall 2020 event, Q1 2021 event, or WWDC 2021. Based on the lack of a more concrete answer, WWDC2020 will likely not see it, but I am open to being wrong here.
Stage3 (Present/2021 - 2022/2023):
Stage 3 involves the first version of at least one fully ARM-powered Mac into Apple’s computer lineup.
I expect this will come in the form of the previously-retired 12” MacBook. There are rumors that Apple is still working internally to perfect the infamous Butterfly keyboard, and there are also signs that Apple is developing an A14x based processors with 8-12 cores designed specifically for use as the primary processor in a Mac. It makes sense that this model could see the return of the Butterfly keyboard, considering how thin and light it is intended to be, and using an A14x processor would make it will be a very capable, very portable machine, and should give customers a good taste of what is to come.
Personally, I am excited to test the new 12" “ARMbook”. I do miss my own original 12", even with all the CPU failure issues those older models had. It was a lovely form factor for me.
It's still not entirely known whether the physical design of these will change from the retired version, exactly how many cores it will have, the port configuration, etc. I have also heard rumors about the 12” model possibly supporting 5G cellular connectivity natively thanks to the A14 series processor. All of this will most likely be confirmed soon enough.
This 12” model will be the perfect stepping stone for stage 3, since Apple’s ARM processors are not yet a full-on replacement for Intel’s full processor lineup, especially at the high end, in products such as the upcoming 2020 iMac, iMac Pro, 16” MacBook Pro, and the 2019 Mac Pro.
Performance of Apple’s ARM platform compared to Intel has been a big point of contention over the last couple years, primarily due to the lack of data representative of real-world desktop usage scenarios. The iPad Pro and other models with Apple’s highest-end silicon still lack the ability to execute a lot of high end professional applications, so data about anything more than video editing and photo editing tasks benchmarks quickly becomes meaningless. While there are completely synthetic benchmarks like Geekbench, Antutu, and others, to try and bridge the gap, they are very far from being accurate or representative of the real real world performance in many instances. Even though the Apple ARM processors are incredibly powerful, and I do give constant praise to their silicon design teams, there still just isn’t enough data to show how they will perform for real-world desktop usage scenarios, and synthetic benchmarks are like standardized testing: they only show how good a platform is at running the synthetic benchmark. This type of benchmark stresses only very specific parts of each chip at a time, rather than how well it does a general task, and then boil down the complexity and nuances of each chip into a single numeric score, which is not a remotely accurate way of representing processors with vastly different capabilities and designs. It would be like gauging how well a person performs a manual labor task based on averaging only the speed of every individual muscle in the body, regardless of if, or how much, each is used. A specific group of muscles being stronger or weaker than others could wildly skew the final result, and grossly misrepresent performance of the person as a whole. Real world program performance will be the key in determining the success and future of this transition, and it will have to be great on this 12" model, but not just in a limited set of tasks, it will have to be great at *everything*. It is intended to be the first Horseman of the Apocalypse for the Intel Mac, and it better behave like one. Consumers have been expecting this, especially after 15 years of Intel processors, the continued advancement of Apple’s processors, and the decline of Intel’s market lead.
The point of this “demonstration” model is to ease both users and developers into the desktop ARM ecosystem slowly. Much like how the iPhone X paved the way for FaceID-enabled iPhones, this 12" model will pave the way towards ARM Mac systems. Some power-user type consumers may complain at first, depending on the software compatibility story, then realize it works just fine since the majority of the computer users today do not do many tasks that can’t be accomplished on an iPad or lower end computer. Apple needs to gain the public’s trust for basic tasks first, before they will be able to break into the market of users performing more hardcore or “Pro” tasks. This early model will probably not be targeted at these high-end professionals, which will allow Apple to begin to gather early information about the stability and performance of this model, day to day usability, developmental issues that need to be addressed, hardware failure analysis, etc. All of this information is crucial to Stage 4, or possibly later parts of Stage 3.
The 2 biggest concerns most people have with the architecture change is app support and Bootcamp.
Any apps released through the Mac App Store will not be a problem. Because App Store apps are submitted as LLVM IR (“Bitcode”), the system can automatically download versions compiled and optimized for ARM platforms, similar to how App Thinning on iOS works. For apps distributed outside the App Store, thing might be more tricky. There are a few ways this could go:
- Developer will need to build both x86_64 and ARM version of their app - App Bundles have supported multiple-architecture binaries since the dawn of OS X and the PowerPC transition
- Move to apps being distributed in an architecture-independent manner, as they are on the App Store. There is some software changes that are suggestive of this, such as the new architecture in dyld3.
- An x86_64 instruction decoder in silicon - very unlikely due to the significant overhead this would create in the silicon design, and potential licensing issues. (ARM, being a RISC, “reduced instruction set”, has very few instructions; x86_64 has thousands)
- Server-side ahead-of-time transpilation (converting x86 code to equivalent ARM code) using Notarization submissions - Apple certainly has the compiler chops in the LLVM team to do something like this
- Outright emulation, similar to the approach that was taken in ARM releases of Windows, but received extremely poorly (limited to 32-bit apps, and very very slow)There could be other solutions in the works to fix this but I am not aware of any. This is just me speculating about some of the possibilities.
As for Bootcamp, while ARM-compatible versions of Windows do exist and are in development, they come with their own similar set of app support problems. Microsoft has experimented with emulating x86_64 on their ARM-based Surface products, and some other OEMs have created their own Windows-powered ARM laptops, but with very little success. Performance is a problem across the board, with other ARM silicon not being anywhere near as advanced, and with the majority of apps in the Windows ecosystem that were not developed in-house at Microsoft running terribly due to the x86_64 emulation software. If Bootcamp does come to the early ARM MacBook, it more than likely will run like very poorly for anything other than Windows UWP apps. There is a high chance it will be abandoned entirely until Windows becomes much more friendly to the architecture.
I believe this will also be a very crucial turning point for the MacBook lineup as a whole. At present, the iPad Pro paired with the Magic Keyboard is, in many ways, nearly identical to a laptop, with the biggest difference being the system software itself. While Apple executives have outright denied plans of merging the iPad and MacBook line, that could very well just be a marketing stance, shutting the down rumors in anticipation of a well-executed surprise. I think that Apple might at least re-examine the possibility of merging Macs and iPads in some capacity, but whether they proceed or not could be driven by consumer reaction to both products. Do they prefer the feel and usability of macOS on ARM, and like the separation of both products? Is there success across the industry of the ARM platform, both at the lower and higher end of the market? Do users see that iPadOS and macOS are just 2 halves of the same coin? Should there be a middle ground, and a new type of product similar to the Surface Book, but running macOS? Should Macs and iPads run a completely uniform OS? Will iPadOS ever see exposed the same sort of UNIX-based tools for IT administrators and software developers that macOS has present? These are all very real questions that will pop up in the near future.
The line between Stage 3 and Stage 4 will be blurry, and will depend on how Apple wishes to address different problems going forward, and what the reactions look like. It is very possible that only 12” will be released at first, or a handful more lower end model laptop and desktop products could be released, with high performance Macs following in Stage 4, or perhaps everything but enterprise products like Mac Pro will be switched fully. Only time will tell.
Stage 4 (the end goal):
Congratulations, you’re made it to the end of my TED talk. We are now well into the 2020s and COVID-19 Part 4 is casually catching up to the 5G = Virus crowd. All Macs have transitioned fully to ARM. iMac, MacBooks Pro and otherwise, Mac Pro, Mac Mini, everything. The future is fully Apple from top to bottom, and vertical integration leading to market dominance continues. Many other OEM have begun to follow in this path to some extent, creating more demand for a similar class of silicon from other firms.
The remainder here is pure speculation with a dash of wishful thinking. There are still a lot of things that are entirely unclear. The only concrete thing is that Stage 4 will happen when everything is running Apple’s in- house processors.
By this point, consumers will be quite familiar with the ARM Macs existing, and developers have had have enough time to transition apps fully over to the newly unified system. Any performance, battery life, or app support concerns will not be an issue at this point.
There are no more details here, it’s the end of the road, but we are left with a number of questions.
It is unclear if Apple will stick to AMD's GPUs or whether they will instead opt to use their in-house graphics solutions that have been used since the A11 series of processors.
How Thunderbolt support on these models of Mac will be achieved is unknown. While Intel has made it openly available for use, and there are plans to have USB and Thunderbolt combined in a single standard, it’s still unclear how it will play along with Apple processors. Presently, iPhones do support connecting devices via PCI Express to the processor, but it has only been used for iPhone and iPad storage. The current Apple processors simply lack the number of lanes required for even the lowest end MacBook Pro. This is an issue that would need to be addressed in order to ship a full desktop-grade platform.
There is also the question of upgradability for desktop models, and if and how there will be a replaceable, socketed version of these processors. Will standard desktop and laptop memory modules play nicely with these ARM processors? Will they drop standard memory across the board, in favor of soldered options, or continue to support user-configurable memory on some models? Will my 2023 Mac Pro play nicely with a standard PCI Express device that I buy off the shelf? Will we see a return of “Mac Edition” PCI devices?
There are still a lot of unknowns, and guessing any further in advance is too difficult. The only thing that is certain, however, is that Apple processors coming to Mac is very much within arm’s reach.
I really enjoyed m4nz
's recent post: Getting into DevOps as a beginner is tricky - My 50 cents to help with it
and wanted to do my own version of it, in hopes that it might help beginners as well. I agree with most of their advice and recommend folks check it out if you haven't yet, but I wanted to provide more of a simple list of things to learn and tools to use to compliment their solid advice.
While I went to college and got a degree, it wasn't in computer science. I simply developed an interest in Linux and Free & Open Source Software as a hobby. I set up a home server and home theater PC before smart TV's and Roku were really a thing simply because I thought it was cool and interesting and enjoyed the novelty of it.
Fast forward a few years and basically I was just tired of being poor lol. I had heard on the now defunct Linux Action Show podcast about linuxacademy.com
and how people had had success with getting Linux jobs despite not having a degree by taking the courses there and acquiring certifications. I took a course, got the basic LPI Linux Essentials Certification
, then got lucky by landing literally the first Linux job I applied for at a consulting firm as a junior sysadmin.
Without a CS degree, any real experience, and 1 measly certification, I figured I had to level up my skills as quickly as possible and this is where I really started to get into DevOps tools and methodologies. I now have 5 years experience in the IT world, most of it doing DevOps/SRE work.
People have varying opinions on the relevance and worth of certifications. If you already have a CS degree or experience then they're probably not needed unless their structure and challenge would be a good motivation for you to learn more. Without experience or a CS degree, you'll probably need a few to break into the IT world unless you know someone or have something else to prove your skills, like a github profile with lots of open source contributions, or a non-profit you built a website for or something like that. Regardless of their efficacy at judging a candidate's ability to actually do
DevOps/sysadmin work, they can absolutely help you get hired in my experience.
Right now, these are the certs I would recommend beginners pursue. You don't necessarily need
all of them to get a job (I got started with just the first one on this list), and any real world experience you can get will be worth more than any number of certs imo (both in terms of knowledge gained and in increasing your prospects of getting hired), but this is a good starting place to help you plan out what certs you want to pursue. Some hiring managers and DevOps professionals don't care at all about certs, some folks will place way too much emphasis on them ... it all depends on the company and the person interviewing you. In my experience I feel that they absolutely helped me advance my career. If you feel you don't need them, that's cool too ... they're a lot of work so skip them if you can of course lol.
- LPI Linux Essentials - basic multiple choice test on Linux basics. Fairly easy especially if you have nix experience, otherwise I'd recommend a taking a course like I did. linuxacademy worked for me, but there are other sites out there that can help. For this one, you can probably get by just searching youtube for the topics covered on the test.
- Linux Foundation Certified System Administrator - This one is a hands on test which is great, you do a screen share with a proctor and ssh into their server; then you have a list of objectives to accomplish on the server pretty much however you see fit. Write a big bash script to do it all, do like 100 mv commands manually, write a small program in python lol, whatever you want so long as you accomplish the goals in time.
- Amazon Web Services certs - I would go for the all 3 associate level certs if you can: Solutions Architect, SysOps Administrator, Developer. These are quite tedious to study for as they can be more a certification that you know which AWS products to get your client to use than they are a test of your cloud knowledge at times. For better or worse, AWS is the top cloud provider at the moment so showing you have knowledge there opens you up to the most jobs. If you know you want to work with another cloud provider then the Google certs can be swapped out here, for example. I know that with the AWS certs, I get offers all the time for companies that use GCP even though I have no real experience there. Folks with the google certs: is the reverse true for you? (genuinely asking, it would be useful for beginners to know).
- Certified Kubernetes Administrator - I don't actually have this cert since at this point in my career I have real Kubernetes experience on my resume so it's kind of not needed, but if you wanted learn Kubernetes and prove it to prospective employers it can help.
Tools and Experimentation
While certs can help you get hired, they won't make you a good DevOps Engineer or Site Reliability Engineer. The only way to get good, just like with anything else, is to practice. There are a lot of sub-areas in the DevOps world to specialize in ... though in my experience, especially at smaller companies, you'll be asked to do a little (or a lot) of all of them.
Though definitely not exhaustive, here's a list of tools you'll want to gain experience with both as points on a resume and as trusty tools in your tool belt you can call on to solve problems. While there is plenty of "resume driven development" in the DevOps world, these tools are solving real problems that people encounter and struggle with all the time, i.e., you're not just learning them because they are cool and flashy, but because not
knowing and using them is a giant pain!
- Linux! - Unless you want to only work with Windows for some reason, Linux is the most important thing you can learn to become a good DevOps professional in my view. Install it on your personal laptop, try a bunch of different distributions, develop an opinion on systemd vs. other init systems ;), get a few cloud servers on DigitalOcean or AWS to mess around with, set up a home server, try different desktop environments and window managers, master a cli text editor, break your install and try to fix it, customize your desktop until it's unrecognizable lol. Just get as much experience with Linux as possible!
- git - Aside from general Linux knowledge, git is one of the most important tool for DevOps/SREs to know in my view. A good DevOps team will usually practice "git ops," i.e., making changes to your CI/CD pipeline, infrastructure, or server provisioning will involve making a pull request against the appropriate git repo.
- terraform - terraform is the de facto "infrastructure as code" tool in the DevOps world. Personally, I love it despite it's pain points. It's a great place to start once you have a good Linux and cloud knowledge foundation as it will allow you to easily and quickly bring up infrastructure to practice with the other tools on this list.
- packer - While not hugely popular or widely used, it's such a simple and useful tool that I recommend you check it out. Packer lets you build "immutable server images" with all of the tools and configuration you need baked in, so that your servers come online ready to start working immediately without any further provisioning needed. Combined with terraform, you can bring up Kubernetes clusters with a single command, or any other fancy DevOps tools you want to play with.
- ansible - With the advent of Kubernetes and container orchestration, "configuration management" has become somewhat less relevant ... or at least less of a flashy and popular topic. It is still something you should be familiar with and it absolutely is in wide use at many companies. Personally, I love the combination of ansible + packer + terraform and find it very useful. Chef and Puppet are nice too, but Ansible is the most popular last I checked so unless you have a preference (or already know Ruby) then I'd go with that.
- jenkins - despite it's many, many flaws and pain points lol, Jenkins is still incredibly useful and widely used as a CI/CD solution and it's fairly easy to get started with. EDIT: Upon further consideration, Jenkins may not be the best choice for beginners to learn. At this point, you’re probably better off with something like GitLab: it’s a more powerful and useful tool, you’ll learn YAML for its config, and it’s less of a pain to use. If you know Jenkins that’s great and it will help you get a job probably, but then you might implement Jenkins since it’s what you know ... but if you have the chance, choose another tool.
- postgres - Knowledge of SQL databases is very useful, both from a DBA standpoint and the operations side of things. You might be helping developers develop a new service and helping with setting up schema (or doing so yourself for an internal tool), or you might be spinning up an instance for devs to access, or even pinpointing that a SQL query is the bottleneck in an app's performance. I put Postgres here because that's what I personally use and have seen a lot in the industry, but experience with any SQL database will be useful.
- nginx - nginx is commonly used an http server for simple services or as an ingress option for kubernetes. Learn the basic config options, how to do TLS, etc.
- docker - Ah, the buzzword of yesteryear. Docker and containerization is still incredibly dominant as a paradigm in the DevOps world right now and it is paramount that you learn it and master it. Be comfortable writing Dockerfiles, troubleshooting docker networking, the fundamentals of how linux containers work ... and definitely get familiar with Alpine Linux as it will most likely be the base image for most of your company's docker images.
- kubernetes - At many companies, DevOps EngineeSite Reliability Engineer effectively translates to "Kubernetes Babysitter," especially if you're new on the job. Container orchestration, while no longer truly "cutting edge" is still fairly new and there is high demand for people with knowledge and experience with it. Work through Kubernetes The Hard Way to bring up a cluster manually. Learn and know the various "primitives" like pods and replicasets. Learn about ingress and how to expose services.
There are many, many other DevOps tools I left out that are worthwhile (I didn't even touch the tools in the kubernetes space like helm and spinnaker). Definitely don't stop at this list! A good DevOps engineer is always looking to add useful tools to their tool belt. This industry changes so quickly, it's hard to keep up. That's why it's important to also learn the "why" of each of these tools, so that you can determine which tool would best solve a particular problem. Nearly everything on this list could be swapped for another tool to accomplish the same goals. The ones I listed are simply the most common/popular and so are a good place to start for beginners.
Programming Languages Any
language you learn will be useful and make you a better sysadmin/DevOps Eng/SRE, but these are the 3 I would recommend that beginners target first.
- Bash - It's right there in your terminal and for better or worse, a scarily large amount of the world's IT infrastructure depends on ill-conceived and poorly commented bash scripts. It's bash scripts all the way down. I joke, but bash is an incredibly powerful tool and a great place to start learning programming basics like control flow and variables.
- Python - It has a beautiful syntax, it's easy to learn, and the python shell makes it quick to learn the basics. Many companies have large repos of python scripts used by operations for automating all sorts of things. Also, many older DevOps tools (like ansible) are written in python.
- Go - Go makes for a great first "systems language" in that it's quite powerful and gives you access to some low level functionality, but the syntax is simple, explicit and easy to understand. It's also fast, compiles to static binaries, has a strong type system and it's easier to learn than C or C++ or Rust. Also, most modern DevOps tools are written in Go. If the documentation isn't answering your question and the logs aren't clear enough, nothing beats being able to go to the source code of a tool for troubleshooting.
Expanding your knowledge
correctly pointed out in their post, while knowledge of and experience with popular DevOps tools is important; nothing beats in-depth knowledge of the underlying systems. The more you can learn about Linux, operating system design, distributed systems, git concepts, language design, networking (it's always DNS ;) the better. Yes, all the tools listed above are extremely useful and will help you do your job, but it helps to know why
we use those tools in the first place. What problems are they solving? The solutions to many production problems have already been automated away for the most part: kubernetes will restart a failed service automatically, automated testing catches many common bugs, etc. ... but that means that sometimes the solution to the issue you're troubleshooting will be quite esoteric. Occam's razor still applies, and it's usually the simplest explanation that works; but sometimes the problem really is
at the kernel level.
The biggest innovations in the IT world are generally ones of abstractions: config management abstracts away tedious server provisioning, cloud providers abstract away the data center, containers abstract away the OS level, container orchestration abstracts away the node and cluster level, etc. Understanding what it happening beneath each layer of abstraction is crucial. It gives you a "big picture" of how everything fits together and why things are the way they are; and it allows you to place new
tools and information into the big picture so you'll know why they'd be useful or whether or not they'd work for your company and team before you've even looked in-depth at them.
Anyway, I hope that helps. I'll be happy to answer any beginnegetting started questions that folks have! I don't care to argue about this or that point in my post, but if you have a better suggestion or additional advice then please just add it here in the comments or in your own post! A good DevOps Eng/SRE freely shares their knowledge so that we can all improve.
TL;DR: Flash your Raspberry Pi 4, plug in an ethernet cable, connect the SSD disk and power up the device to turn the Raspberry Pi 4 into a full Ethereum 1.0 node or an Ethereum 2.0 node (beacon chain / validator) submitted by
Some background first. As you know, we’ve been running into some memory issues  with the Raspberry Pi 4 image as Raspbian OS is still on 32bits  (at least the userland). While we prefer to stick with the official OS we came to the conclusion that, in order to solve these issues, we need to migrate to a native 64 bits OS
Besides, Eth 2.0 clients don’t support 32 bits binaries so using Raspbian would exclude the Raspberry Pi 4 from running an Eth 2.0 node (and the possibility of staking).
So, after several tests we are now releasing 2 different images based on Ubuntu 20.04 64bit : Eth 1.0 and Eth 2.0 editions.
Basically, both are the same image and include the same features of the Raspbian based images. But they are setup for running Eth 1.0 or Eth 2.0 software by default Images take care of all the necessary steps
, from setting up the environment and formatting the SSD disk to installing and running the Ethereum software as well as starting the blockchain synchronization.
- Based on Ubuntu 20.04 64bit
- Automatic USB disk partitioning and formatting
- Adds swap memory (ZRAM kernel module + a swap file) based on Armbian work 
- Changes the hostname to something like “ethnode-e2a3e6fe” based on MAC hash
- Runs software as a systemd service and starts syncing the Blockchain
- Includes an APT repository for installing and upgrading Ethereum software
- Includes a monitoring dashboard based on Grafana / Prometheus
Both images include the same packages, the only difference between them is that Eth 1.0 runs Geth by default and Eth 2.0 runs Prysm beacon chain by default. Ethereum 1.0 clients
Ethereum 2.0 clients
- Geth : 1.9.13 (official binary)
- Parity : 2.7.2 (cross compiled)
- Nethermind : 1.8.28 (cross compiled)
- Hyperledger Besu : 1.4.4 (compiled)
- Prysm : 1.0.0-alpha6 (official binary)
- Lighthouse : 0.1.1 (compiled)
- Swarm : 0.5.7 (official binary)
- Raiden Network : 0.200.0~rc1 (official binary)
- IPFS : 0.5.0 (official binary)
- Statusd : 0.52.3 (compiled)
- Vipnode : 2.3.3 (official binary)
INSTALLATION GUIDE AND USAGE
Recommended hardware and setup
- Raspberry 4 (model B) - 4GB
- MicroSD Card (16 GB Class 10 minimun)
- SSD USB 3.0 disk (see storage section)
- Power supply
- Ethernet cable
- 30303 Port forwarding (Eth 1.0) and 13000 port forwarding (Eth 2.0) 
- A case with heatsink and fan (Optional but strongly recommended)
- USB keyboard, Monitor and HDMI cable (micro-HDMI) (Optional)
You will need and SSD to run the Ethereum clients (without an SSD drive there’s absolutely no chance of syncing the Ethereum blockchain). There are 2 options:
- Use a USB portable SSD disk such as the Samsung T5 Portable SSD.
- Use a USB 3.0 External Hard Drive Case with a SSD Disk. In our case we used a Inateck 2.5 Hard Drive Enclosure FE2011. Make sure to buy a case with an UAS compliant chip, particularly, one of these: JMicron (JMS567 or JMS578) or ASMedia (ASM1153E).
In both cases, avoid getting low quality SSD disks as it is a key component of you node and it can drastically affect the performance (and sync times)
Keep in mind that you need to plug the disk to an USB 3.0 port (blue)
Image download & installation 1.- Download Eth 1.0 or Eth 2.0 images: ubuntu-20.04-preinstalled-server-arm64+raspi-eth1.img.zip
sha256 34f105201482279a5e83decd265bd124d167b0fefa43bc05e4268ff899b46f19 ubuntu-20.04-preinstalled-server-arm64+raspi-eth2.img.zip
sha256 74c0c15b708720e5ae5cac324f1afded6316537fb17166109326755232cd316e 2.- Flash the image
Insert the microSD in your Desktop / Laptop and download the file (Eth 1.0, for instance):
Note: If you are not comfortable with command line or if you are running Windows, you can use Etcher (https://etcher.io
Open a terminal and check your MicroSD device name running:
sudo fdisk -l
You should see a device named mmcblk0 or sdd. Unzip and flash the image:
unzip ubuntu-20.04-preinstalled-server-arm64+raspi-eth1.img.zip sudo dd bs=1M if=ubuntu-20.04-preinstalled-server-arm64+raspi-eth1.img of=/dev/mmcblk0 && sync 3.- Insert de MicroSD into the Raspberry Pi 4. Connect an Ethernet cable and attach the USB SSD disk
(make sure you are using a blue port). 4.- Power on the device
The Ubuntu OS will boot up in less than one minute but you will need to wait approximately 10 minutes
in order to allow the script to perform the necessary tasks to turn the device into an Ethereum node and reboot the Raspberry.
Depending on the image, you will be running:
5.- Log in
- Eth 1.0: Geth as the default client syncing the blockchain
- Eth 2.0: Prysm as default client syncing the beacon chain (Topaz testnet)
You can log in through SSH or using the console (if you have a monitor and keyboard attached)
User: ethereum Password: ethereum
You will be prompted to change the password on first login, so you will need to login twice. 6.- Open 30303 port for Geth and 13000 if you are running Prysm beacon chain.
If you don’t know how to do this, google “port forwarding” followed by your router model. 7.- Getting console output
You can see what’s happening in the background by typing:
sudo tail -f /valog/syslog
Congratulations. You are now running a full Ethereum node on your Raspberry Pi 4.
Syncing the Blockchain
Now you need to wait for the blockchain to be synced. In the case of Eth 1.0 This will take a few days depending on several factors but you can expect up to about 5-7 days.
If you are running the Eth 2.0 Topaz tesnet you can expect 1-2 days of Beacon chain synchronization time. Remember that you will need to setup the validator later in order to start the staking process (see “How to run the Eth 2.0 validator” section below).
For this first release, we included 3 monitoring dashboards based on Prometheus  / Grafana  in order to monitor the node and clients’ data (Geth and Besu). You can access through your web browser:
URL: http://your_raspberrypi_IP:3000 User: admin Password: ethereum
All clients run as a systemd service. This is important because in case of some problem arises the system will respawn the process automatically.
Geth and Prysm beacon chain run by default (depending on what you are synchronizing, Eth 1.0 or Eth 2.0) so, if you want to switch to other clients (from Geth to Nethermind, for instance), you need to stop and disable Geth first, and enable and start the other client:
sudo systemctl stop geth && sudo systemctl disable geth
Commands to enable and start each Eth 1.0 client:
sudo systemctl enable besu && sudo systemctl start besu sudo systemctl enable nethermind && sudo systemctl start nethermind sudo systemctl enable parity && sudo systemctl start parity
sudo systemctl stop prysm-beacon && sudo systemctl disable prysm-beacon sudo systemctl start lighthouse && sudo systemctl enable lighthouse
Clients’ config files are located in the /etc/ethereum/ directory. You can edit these files and restart the systemd service in order for the changes to take effect. The only exception is Nethermind which, additionally, has a mainnet config file located here:
Blockchain clients’ data is stored on the ethereum home account as follows (note the dot before the directory name):
/home/ethereum/.geth /home/ethereum/.parity /home/ethereum/.besu /home/ethereum/.nethermind
/home/ethereum/.eth2 /home/ethereum/.eth2validators /home/ethereum/.lighthouse Hyperledger Besu and Nethermind
Nethermind and Hyperledger Besu
These 2 great Eth 1.0 clients have become a great alternative to Geth and Parity. The more diversity in the network, the better, so you may give them a try and contribute to the network health.
Both need further testing so feel free to play with them and report back your feedback.
How to run the Eth 2.0 validator (staking)
Once the Topaz testnet beacon chain is synchronized you can run a validator in the same device. You will need to follow the steps described here: https://prylabs.net/participate
The first time, you need to create manually an account by running the “validator” binary and setup a password. Once you completed this step you can add the password to /etc/ethereum/prysm-validator.conf and start the validator as a systemd service
We put a lot of work trying to setup the Raspberry Pi 4 as a full Ethereum node as we know the massive user base of this device may have a very positive impact in the network.
Please, take into account that this is the first image based on Ubuntu 20.04 so there may be some bugs. If so, open an issue on Github or reach us on twitter (https://twitter.com/EthereumOnARM