Back in February 2019, while referring to Arm server, Linus Torvalds famously said:
I can pretty much guarantee that as long as everybody does cross-development, the platform won’t be all that stable. Or successful.
…If you develop on x86, then you’re going to want to deploy on x86, because you’ll be able to run what you test “at home” (and by “at home” I don’t mean literally in your home, but in your work environment).
…
Which in turn means that cloud providers will end up making more money from their x86 side, which means that they’ll prioritize it, and any ARM offerings will be secondary and probably relegated to the mindless dregs (maybe front-end, maybe just static html, that kind of stuff).
SolidRun had already worked on products with NXP LX2160A 16-core Arm Cortex A72 processor and found out it could be a match to make a powerful Arm workstation so that code could be developed natively on Arm like on a standard (x86) PC, and by the end of March 2019 announced plan for “ClearFog ITX Workstation“. The name of their product eventually became HoneyComb LX2K, a mini-ITX board fitted with the company’s CEx7 LX2160A COM Express module powered by NXP LX2160A SoC, and pre-production models went for pre-order for $550 in May 2019, with shipping several months later. HoneyComb LX2K is now available for purchase for $750.
The board comes with 64GB eMMC flash, but to create a complete Arm workstation, you’ll need to add DDR4 memory sticks, optional extra storage via SATA ports or the M.2 socket, a PCIe graphics card, and a Mini-ITX case with a power supply which may look like the one shown above. The system is said to consume just around 15 watts at idle. I took the workstation from a video just uploaded by SolidRun where you can learn more.
HoneyComb LX2K 64-bit Arm workstation is primarily designed for developers who can natively develop Armv8 applications in Linux as they would for x86 programs in a PC thanks to a powerful 16-core Cortex-A72 processor and support for up to 64GB RAM. Solidrun is also working on ACPI support and SBSA compliance. You can find hardware and software documentation on the developer website.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
modprobe binfmt_misc ; cp -a /usr/bin/qemu-aarch64-static ; chroot /bin/sh
Cant beat that … 🙂
p.s except when java vm is inside the chroot
then again who needs java
I don’t believe this will become a successful development workstation. The first thing is that it is extremely expensive compared to what you get from x86 devices which achieve roughly 4 times the same single-core performance, which still remains the dominant factor for every day development (./configure, git, browser, IDE, etc). Second, when you listen to users of non-linux operating systems, they often complain that one specific driver is missing on their OS or that some software is only provided for Linux, and these ones will not work there either. It may sometimes be something as stupid as a multi-function printer driver, a browser plugin, a video codec or a development utility (such as Rockchip’s upload utilities), that will make these developers’ life irritating.
I’m a long time user of cross-compilation, and I don’t find it difficult at all. Moreover it often helps detecting undesired dependencies that ruin your life in production when something only works on your development machine. However I also value native development when it comes to quick hacking or testing. Typically when optimizing some code for performance, you sometimes prefer to modify, rebuild and run on the target than do it on the development machine followed by scp or an NFS share.
With a much lower price (and quite a smaller and FANLESS enclosure) this machine could become a second development machine lying on the desk. But not at that price and size (and possibly noise given it has at least three fans, which is already two more than in my PC).
It is a standard mini-ITX form factor for this reason. Build it how you want. Put it in a 1RU. I have mine watercooled with an AIO loop and it is whisper quiet. We like the H200i case because it is open and allows you to see the board, not because it is the perfect case for everyone.
As for the other bits and pieces missing. Yes there are rough edges, but nobody is going to fix those issues until there is a reason to. We are trying to help create the reason. This isn’t a board for everyone, and that is why we consider it a workstation and not a PC. Not everyone needs 16 cores, 64GB of memory, and multiple 10Gbps network interfaces, but for those that do we are providing a solution that is far below the cost of the other options out there. This is not a gaming PC, but a tool for those that need it.
Oh don’t get me wrong Jon, I receive these arguments. But quite frankly the price-to-benefit seems way too high for this use case and I suspect that the original area of expertise of the chip is largely responsible for this. I mean, as a workstation, a single 10G or even 1G would be enough. The 10/100G ports are amazingly interesting to build network gear and I even wondered if we couldn’t use your SoM in our LBs (except that in the end it’s limited to 40G and it’s not that interesting anymore). Probably that a scaled down version of the chip with much less I/O at half the price would be more appealing to developers (but I know you cannot decide on the SoC’s features and price of course).
And to make things clear, I never compared this to a gaming machine, I’m not interested by these and do not even have a gaming PC. You just said “for those that need it” but my suspicion is that this public is quite tight at this price tag, but surely it does exist, just like I wouldn’t be surprised if some developers install a graphics card on their MacchiatoBin (mine is headless :-)).
I think the price is fair for what it is providing. This is a chip that is focused on specific features that some people don’t care about and we understand that. If you look at the price of an x86_64 equivalent workstation / server grade CPU I think you will see it is quite similar. This is not something that should be compared to your i7 / Ryzen CPUs, but instead your low end Xeon and Epyc offerings. Support for ECC DDR4 3200, multiple 10Gbps networking, Sata with ECC, are all features that you find only on Workstation / Server grade hardware and they are priced much higher because of it. Even with all those features we are still less than half the price of other ARM64 workstation options.
As I said, for a server it’s a nice board and SoC, and the price is even quite correct. But few people need that on a workstation IMHO, which sadly inflates its cost. I’d be glad to be proven wrong because I’d really love to see you continue to succeed on high-end SoCs!
This is not a consumer PC, and does not benefit from the same economy of scale. If you need an arm workstation (because e.g. you’re porting a massive codebase), a ryzen does not cut it somehow.
There is no hardware where there’s no driver for.
If you have a windows driver, you can import it in Linux. If you have a red hat driver, you can convert it to Debian.
This unit has 2 things against it,
1- Price.
No one buys this, to save money, because even at a generous low 20 Watts at full load, it would take 10 years to pay back the difference with an x86 hardware in terms of cost of electricity.
2- performance.
I bet you, the unit uses 15W idle, but under load will be close to 20-25W.
For industrious use, they’d either have to increase the core speed to 3,5Ghz (with ~10W higher consumption), or increase the cores.
For the price, and CPU speed, I’d expect an arm machine like this with at least 64 threads.
But even that would use less than 50W.
If they want to be successful, make a simple board like this, with 4 (registered) ddr4 ram slots, and using 128 arm cores or threads, and sell it (street price) for $1000-1200.
A unit like that could be using 25-35W idle, but no more than 100W under load. And that’s what makes it interesting!
128 Watts, 128 cores at 2 Ghz, would best any x86 in performance per watt!
For that price, I can either get a ryzen 3900x with Mobo and ram, or get 25 quadcore AMLogic tv boxes.
I would have superior processing power either way.
Yes you can if that is what you want to spend your money on. Neither of those are solutions we are trying to replace.
I’ll get one, but I won’t scrap my x86 workstation. I spent a couple of weeks using a pinebook pro for work and play. Work was broadly fine (docker, kubectl, terraform etc) but I’d have to get comfy with Qemu for a few key programs like Postman.
Play is a non-starter. Even Minecraft has an x86 only launcher while the game itself is Java.
Well there are options. Android games running under SPURV are something I have been playing around with. Alternatively Cloud gaming could be an option if you are into that.
Looks more like a cheap Dell small form factor desktop than a “workstation”.
Interesting machine. I mean: Yes, expensive at the moment. But still interesting. When the price drops, this machine is actually a really great thing. Especially the SFP+ ports are helpful in some situations.
I’m not holding my breath on price drops, at least not until this is EOL’ed…
I don’t think the price will drop because the SoC itself is rather made for high performance networking gear (100G ethernet is supported for example), and the price is totally fine when you use it to build a new router or load balancer. It’s just overkill for a development machine. I’d rather get a quad-A77 at 3 GHz from a smartphone SoC, it would be much cheaper and likely faster on most workloads!
Or just buy a dell desktop to cross compile with from your windows laptop running putty like almost everyone working on linux for ARM is doing.
I think there are some rough edges you will only discover and then solve through usage of a real device. Take Linux on a phone for instance, you can cross compile apps and test them in an emulator but it would take real hardware to discover a bug that stops you making a phone call.
>I think there are some rough edges you will only discover
>and then solve through usage of a real device.
That doesn’t mean you have to develop on the real device. In a lot of debugging situations the target device isn’t even usable for console output during debugging (CPUs halted) and you have to remotely debug via JTAG or whatever… This is basically one of the advantages (low level debugging mostly works) of using ARM so it’s really weird that people here don’t seem to get that.
Linus is right that ARM is second fiddle but wrong about it being because people are cross compiling. It’s second fiddle because aside from hobbyists no one actually gives a shit about Linux itself in that ecosystem and the lack of giving a shit results in the horrible vendor kernels, GPL violations etc etc. Linux is a means to an end (We want networking, filesystems etc but don’t want to write/license it) not something they are personally invested in.
I’m never working on emulators, always on real hardware. When you play with network you have no option. And very often the hardware that is very network capable shows terrible performance for development. So what you do in this case is develop on your regular PC using your regular tools, IDE etc, and scp the binary to the ARM machine, or boot it off bootp+tftp etc, and test it for real. I love the MacchiatoBin precisely because it’s the first machine I’m seeing which shows exceptional network performance with reasonably good CPU performance. So it is possible to occasionally build on it without waiting too much, but even then most of the time you’d build on your PC and upload the result to get a faster result.
>So what you do in this case is develop on your regular PC using your regular tools,
>IDE etc, and scp the binary to the ARM machine, or boot it off bootp+tftp etc, and test it for real.
Setup CI. A few hours of work will shave hours off of your development routine. Once you have CI you can use the “push any old crap to git and rebase it later” development model that allows you to work even from your smart phone while taking a poo. No expensive workstation needed.
The development is not about cross-compiling a C program or kernel to run on your SBC. It is about having a local development workstation that reflects what you want to deploy in the cloud. Being able to replicate a problem locally in a python script, a node.js app, a wordpress plugin that you want to deploy on an Amazon Gravitron2 instance in the cloud is very useful. You can invest in local comparable hardware and not have to pay Amazon while you fix problems, then push natively to their instances.
If you’re working on something that will be deployed to AWS then you probably depend on stuff that is only available inside of AWS. Unless you can get a machine that is just like the one in AWS including all of the SaaS etc around it then I don’t get your point… also what stops you from developing on any old box and pushing to a regular ARMv8 SBC if you can run it all locally? Also $1000 goes a long way on AWS. If you’re spending that sort of money on AWS debugging then I think you might need a lot more money to create a similar environment locally.
TL;DR; you’re basically saying that somehow bugs happen due to where your source code is typed.
I think you missed the part where I pointed to interpreted languages that you would want to test locally, especially since the support for ARM64 is relatively new especially at a scale of 16+ cores. This is the point that Linus was making when he said that Arm servers wouldn’t take off until there were workstations for developers. If you are using a specific architecture and running it locally you will feel more confident in the overall software stack and how it will perform because you are developing on it locally before deploying on the cloud. If you write a piece of code and test it on x86_64 and then push it to an Aarch64 server you have no idea how it will work regarding possible bottlenecks and bugs. This becomes even more relevant when you are doing micro-optimizations, SIMD, and scaling.
We are not forcing this as a solution to anyone. If you are happy with your development process then all the power to you. However many developers have been happy we are providing this platform and we are excited about the community we are building around the product.
Just to clear something up. A single a1.2xlarge 8vcpu 16gb RAM costs $1782.144 excluding storage for a year of service. This is the original Gravitron not the Gravitron2. If you want to build a local infrastructure for development and testing before deployment then pricing wise the HoneyComb or ClearFog LX2K are quite price competitive. We are also quite workload competitive. https://openbenchmarking.org/result/1905316-JONA-181212252
>A single a1.2xlarge 8vcpu 16gb RAM costs $1782.144 excluding storage for a year of service.
That’s a year running 24/7 right? So if you need to debug something you have running on AWS somewhere else you might need it for a few hours a week… so you probably won’t be paying for a year. You’ll also have the advantage that it’s the same environment as you are actually targeting, has all of the same AWS services, works with your existing EB deployment setup etc etc.
Your numbers there indeed look really decent, I think you did a great job on that board. Do you think we’ll eventually see a working 100G version of it ? That’s what tempted me the most initially but I got cold-showered when seeing that in the end the 100G is limited to 40G for now.
>Linus was making when he said that Arm servers wouldn’t take off until there were workstations
The point Linus was making was that until you are sitting in front of an ARM machine, using it for everything and getting pissed off enough with all of the issues to actually fix them then it’ll never work as well as X86 which has had decades of people sitting in front of it, getting pissed off with it enough to actually fix it. The thing he got wrong is that issue exists due to cross compiling instead of 90%+ of ARM development being based on the “download linux, hack it until it works, ship it, go bust” development model.
>If you are using a specific architecture and running it locally you will feel more
>confident in the overall software stack and how it will perform because you are
>developing on it locally before deploying on the cloud.
Come off it. Developers shouldn’t care where they edit text. If you need a specific arch to edit text your workflow is broken/dumb.
>If you write a piece of code and test it on x86_64 and then push it to
>an Aarch64 server you have no idea how it will work regarding possible bottlenecks and bugs.
If you write some code and run it on the same machine running all of your other junk how do you know that the other junk isn’t interfering with your code?
>This becomes even more relevant when you are doing micro-optimizations, SIMD, and scaling.
You want a clean environment to do that not your desktop running tons of other junk.
Obviously you really don’t want to use a non Intel desktop. Nobody is forcing this tool on you. This is just a choice that is available if people prefer to use it.
>Obviously you really don’t want to use a non Intel desktop.
I don’t care if my desktop is Intel. If AMD got their act together and put some effort into their linux support I’d love to spend almost nothing on a powerful machine with tons of cores, memory and IO up the wazoo . I’m certainly not going to pay a premium to constantly fight against a machine that only sorta works as a desktop to be able to say “hey look at my vim/emacs/nano/eclipse.. notice what’s different? Well my $editor runs on 100% vegan ARM instructions so it’s totally better than yours… somehow”.
>Nobody is forcing this tool on you.
This is a discussion. You’re saying “you really need an ARM desktop like this to do development” and I’m saying “no, the people I work with on a daily basis that are putting linux on ARM into real products aren’t thinking they really must have an ARM workstation do do this job I’ve done for years with a windows laptop and putty”.
Nobody is saying that you really need an ARM desktop to do development. We are saying if you would like to have an ARM64 workstation to do development on, then we have a product for you. There are plenty of developers that would prefer this option but haven’t had an option, now they have an option.
>There are plenty of developers that would prefer
>this option but haven’t had an option, now they have an option.
I doubt it highly. There’s no compelling reason why you would care about what instruction set your text editor runs on. Being able to connect multiple fast storage devices, big displays and so on is much more important.
You’re making this thing and the best you could come up with was making your life more difficult by just not spinning up another instance in EB…
This is a network appliance SoC in a PC case not something like the SPARC Stations or HP workstations of yore where you were paying a premium for something that was considerably more powerful and better made than a PC clone.
In the current climate of not being able to go to the office strapping your work flow to an expensive unicorn machine seems even more ridiculous. I can just see the standup meetings now:
Boss: Dave what did you do today?
Dev: Nothing, I can’t do anything unless I’m physically in front of a machine that only uses locally sourced vegan instructions…
Boss: Why can’t you just SSH into one of the many development targets we have like everyone else?
Dev: That would be against my irrational, borderline mental illness level, hatred for things that just work since that sales rep from Intel ran over my dog after sleeping with my wife sir.
Boss: HR will be in touch.
So you are upset because this isn’t a portable solution? Why can’t you ssh into your workstation remotely and use it like that if you have a laptop?
This is a workstation because it includes high speed storage options, and high speed networking options. You can have a 10Gbps connection to a high speed storage device using NVMe over IP, and another 10Gbps connection to a local build cluster. You can run multiple VMs for testing a native Android app, or Windows on ARM.
This obviously doesn’t fit into what you develop on, but it is not restricting workflow anymore than an x86_64 workstation would that isn’t mobile. If you don’t need these options the product isn’t for you, but it isn’t like there isn’t anyone that wouldn’t like these features.
>So you are upset because this isn’t a portable solution?
I’m not upset about anything..
>Why can’t you ssh into your workstation remotely
>and use it like that if you have a laptop?
If I’m going to do that why does said workstation *need* to be this and not a cheaper generic Dell or HP box?
>This is a workstation because it includes high speed storage options
>, and high speed networking options.
Ugh, no. You have one expansion slot that is used once you put a GPU in it.. and one slot for a local SSD. So this is no better than a small form factor Dell. An old SPARC Station could take multiple drives, multiple network cards, framebuffers etc. This has a whole 2 USB connectors on board when even the crapest SFF machine you can get has 4 or 6 on the back and more the front.
>You can have a 10Gbps connection to a high speed storage device using NVMe over IP,
The 10Gbps networking is literally the only stand out thing this has aside from the number of cores. But it’s pretty normal for a networking SoC to have good networking… Anyhow, so now I need to spend extra for this workstation and a X86 box with 10Gbps networking to host my storage instead of just buying any cheap motherboard from Asus etc that can house multiple NVMe SSDs on the board itself? This seems to be going really backwards really quickly. All this trouble just to avoid one or two commands with the AWS cli to push something onto their ARM offerings.. you obviously have more time to burn faffing around and not doing any actual work than I do.
>and another 10Gbps connection to a local build cluster.
So I need this and a local build cluster now… instead of any recent generic PC with 32GB or 64GB of RAM and a bunch of SSDs which I can buy off of Amazon with next day shipping and get replacements next day if it doesn’t work?
>You can run multiple VMs for testing a native Android app, or Windows on ARM.
Or I could just run the Android app in the KVM based emulator like literally every Android dev does?
>This obviously doesn’t fit into what you develop on,
I didn’t say that. I said this has no advantage for any situation you could name over a generic box from Dell. The edge cases like needing to tests something on ARMv8 on a bunch of cores isn’t going to make this sell like hot cakes.
>anyone that wouldn’t like these features.
This has two *features*: High number of cores and decent networking. If you’re an ARM fanatic maybe you could add that it’s ARMv8 too. On the other hand it has all of these demerits: It’s expensive, the expansion is very limited, it’s made by a small company, linux on ARM is a ghetto and so on and so on…
“Developers shouldn’t care where they edit text”
I’d much rather edit text on my local machine than on a remote one (latency). I’d much rather edit it on my machine with my configuration than another without it. If I’m in the code, build, test, repeat cycle I want that loop to be as tight as possible so I try to build locally unless my local machine has an Atom 230 or something. Now let’s imagine I’m working on stuff that requires ARM64… You get the picture.
>I’d much rather edit text on my local machine than on a remote one (latency).
Can you name a text editor or IDE that only runs on ARM64 or produces better ASCII/UTF8 encoded text on ARM64?
>If I’m in the code
So you want to code on a capable machine that doesn’t stutter like crazy when indexing your massive code base. You might also want to be able to drive two or more 4K displays without the system crapping itself. Commodity gaming PC with an nvidia GPU will do that with enough juice left for youtube k-pop videos playing in the background.
> build,
Again, this clocks at 2ghz. Unless your code base can be built entirely in parallel to utilise the 16 cores that’s going to feel slower than anything from intel or amd in the last few years turboing at 4ghz or more. This is a networking SoC. The low clock speed doesn’t matter so much because each core should be munching away on isolated flows of data. In comparison most build processes have many serialisation points where single threaded brute force is the only solution.
>test, repeat cycle I want that loop to be as tight as possible
I used to build a big android app probably 50 or more times in a day. Pushing the 100MB APK to the emulator or a real target over ADB was a fraction of the whole process. Doing that on a machine like this would make the build take 3 or 4 times as long if it would even complete and then still require the same amount of time to deploy because the vast majority of that process is installing the APK which would be required even if deployed locally.
>Now let’s imagine I’m working on stuff that requires ARM64
>You get the picture.
Lets not imagine. Lets come up with a concrete example of where you must have the development, build and test cycle fully contained on a single ARM64 desktop/workstation instead of doing the sane thing of cross compiling and deploying to a specific target in CI after doing cheap testing locally.
I have to say I agree with dgp here and what he describes follows my workflow as well. I always cross-compile, even x86 to x86, because 1) it’s the only way to keep a controlled environment (i.e. not accidently depend on a crappy lib you installed the previous day for a shiny new editor), and 2) this allows me to use distcc and distribute the work over our MacchiatoBin load generators and cut the build time in 3 compared to a local build. Certain days it saves up to one hour and a half of pure wait time (and it’s not affected by your browser eating 2 cores, nor does it have any performance impact on your editor).
The only times I perform a native build is to build and install stuff on my local machine. It’s so infrequent that I never know my compiler version 🙂
look at this:
https://www.cnx-software.com/2019/11/29/huawei-kunpeng-desktop-board-kunpeng-920-arm-server-processor/
Or due to the Coronavirus down turn you wait and see if you can get a intel i9 at knock down price?
Brutal I know 🙁
https://www.aliexpress.com/item/4000354145934.html?spm=2114.12010612.8148356.2.524459b5ogQEmz
pffft. If it’s not big endian then it’s not worth bothering with. How do developers know if they have bugs if they aren’t using the one true endian?
Hi, what is the LX2’s idle wat consumption approx?
I can’t see it anywhere. A Xeon server should idle around 100-200W I guess, ARM64 systems including Apple M1 should be less.
The last ‘Xeon servers’ installed at a customer idled at 65W each including eight SATA SSDs as RAID-6 behind an internal Dell PERC RAID controller. No energy inefficient networking, just 3 x GbE (iDrac set to 100 MBit/sec).
A MacBook Air (ARM64 Apple M1) idles at 0.5W with disabled LCD backlight.
And ofc no idea about LX2’s idle consumption 😉