I am the engineer at System76 currently working on this. We are using ME cleaner with -S on all systems where possible - HAP bit will be set AND code removed. All systems will then be tested thoroughly in this configuration before it is released to customers.
Relevant source code can be found in the following places, keep in mind that it is still work in progress:
Next time I consider a laptop, you made it tot the top of my list, even if it costs more.
Hear that, Intel? I will put money where my mouth is. I am not sure I can afford any of these RISC-V chips and they are still alpha quality beyond the Talos, but those of my ilk will do their damndest to make you pay for tone deaf reaction.
So IF they had a Power0 chip it wouldn't be Intel. Though I have zero idea how System76 would put Power9 into a workstation laptop but they do put a desktop cpu into their Serval lineup. https://system76.com/laptops/serval
Counterpoint: I bought a System76 laptop roughly eight (?) years ago (Bonobo Pro 17 in.) and I have no plans to buy from them again.
My system was delivered with a BIOS issue that made it only boot perhaps 1 out of 30 cycles. Delivered on a Thursday night, after spending a lot of time troubleshooting, I had to wait until their support got in on Monday to get anything done. Then I had to mail it in and wait a long time to get it returned with a reflashed BIOS (I'd be lying if I said I remember exactly how long, just remember it feeling way too long).
I explained that I had bought the laptop specifically for on-site work with a client and that I needed a functioning fix or replacement ASAP. They told me they couldn't make anything happen any faster, and even told me that they had no way for me to pay out of my own pocket for faster return shipping, though IMO they should've offered to do this themselves.
I did eventually get back a mostly-working system which I still use around the house, though it's had a habit of hard-locking when the GPU is under stress the whole time I've had it. But my experience with Sys76 customer support is the biggest factor in deciding not to buy from them again.
Do bear in mind this was several years ago and that it is reported from memory. It is entirely possible they've gotten their act together and/or that I'm misremembering some details, but this was my experience as I recall it.
>>> Do bear in mind this was several years ago and that it is reported from memory. It is entirely possible they've gotten their act together and/or that I'm misremembering some details, but this was my experience as I recall it.
Not as good as I want it to be. When we manufacture our laptop chassis here in Denver, Colorado we will have more control of battery size and will always prefer longer battery life
> Next time I consider a laptop, you made it tot the top of my list, even if it costs more.
I won't, if it means having to run Ubuntu (and derivatives) or Pop. I'd like to run the distro of my choosing, thanks, but no thanks. It will be possible to disable ME on other laptops/desktops as well.
You can run whatever distro you want, and the hardware issues should minimal at most? The nice thing is that the hardware is supported by the kernel and you won't run into a hardware issue like we use to all the time 10 years ago.
You cannot run whatever distro you want on a System76 if you want ME disabled.
> You must run Ubuntu 16.04 LTS, Ubuntu 17.04, Ubuntu 17.10, Pop!_OS 17.10, or an Ubuntu derivative and have the System76 driver installed to receive the latest firmware and disabled ME on laptops
My guess is just a QA burden: they probably test the firmware cleaner utility on those operating systems, and they don't have the resources to test and certify the firmware tool on every conceivable distro.
Maybe it will run fine on Fedora, but if it bricks your laptop, they don't want to get blamed.
I'm not sure their motives for this choice, as their means of disabling as much of ME as possible is ME cleaner, which is not tied to distros.
I don't see why you can't attempt to do it manually yourself, without the aid of their driver and firmware update tool. Except that you might forfeit any warranties/support.
System76, can you clarify?
Sadly it's standard to only see official support for Ubuntu or RHEL in the Linux world. Not that I don't understand, it's all very fragmented, and that creates a lot of work as to support multiple distros. People are working on remedies, though.
? Unless you intend to fab your own RISC-V processor, once they become available, they will most certainly be priced competitively (or they won't sell). The first models will be available in 2018 and more will follow.
I have been skimming too light on the market details for that bad boy. Only proves the point that the high-cost might be the price of helping proprietary hardware feel the sting of changing cultural tides.
Talos II is POWER9. POWER also has an open design, so it's effectively the same from a freedom perspective as RISC-V (with the distinction that you can buy POWER chips today, but you can't buy RISC-V ones).
Just don't develop any hardware problems or you'll get stuck in a customer support loop around uninstalling and installing nvidia drivers. I haven't been able to use my oryx for about a year, and each time I ask support (the display doesn't turn on) they just advise removing and re-installing the nvidia drivers.
System76 employee here. Sorry to hear about the continued issue with your computer. I've started up the conversation again in the existing ticket. We will get to the bottom of this problem, and get your system working again.
Not currently. I have experimented with coreboot but did not get far yet. We are tied into AMI for most machines right now, and I would love to cut them out and go open source!
We may be experimenting with coreboot more in the near future
FYI, there is a good technical reason in additional to the philosophical one.
I used to work on embedded systems where I have full control of uboot. With that, I can easily reserve the section of memory in boot uboot and Linux kernel when the keys logging info (such as FTRACE of ISR, sched switches etc) from kernel are kept.
When a kernel panic happens, the system will go thru the warmboot sequence and detected there are logging info in that section of DDR and dump it out before continue to boot.
This "postmortem debugging features" make debugging certain category of very difficult bugs (kernel driver panic) much easier.
With typical close source x86 bios in Linux laptop, it is very hard to enable this kind of debugging.
Would you consider partnering with someone like Dell as a supplier?
I've written this elsewhere, but I'll repeat: it seems a shame that folks like System76, Purism, aspiring Kickstarter campaigns et al aren't leveraging the supply chain already in place for Chromebooks. In the case of Purism, for example, x86-64 Chromebooks comparable to Purism's low end models can be had from Dell for 3/4 the asking price of the former. Considering that the driver situation is necessarily a solved problem for Chromebooks, and they're already using coreboot, it seems silly to start a parallel effort to deal in hardware the way System76 and Purism are doing, rather than just approaching Dell about a white label deal to supply you with hardware components that are already known to work.
Alienware went this route and they turned then into mostly generic video card boosted mediocrity.
The business model of system76 was what I was personally looking for and I've been very happy with my latest laptop from them. For me personally, the luster would have been lost coming from Dell.
Dell makes lovely linux-laptops, but I don't think they run coreboot. Partnering with Google for the chromebook supplychain seems like the better theory/plan.
A white-label, more configurable, plain ubuntu version of the pixelbook sounds like a dream.
> Dell makes lovely linux-laptops, but I don't think they run coreboot.
Dell makes Chromebooks. These Chromebooks run coreboot. Ergo, Dell makes laptops that run coreboot.
There's nothing in my comment about building off the back of Dell's existing program to sell Ubuntu machines to customers—a program whose success story is reported to be bizarrely hit-or-miss with respect to driver issues, even.
Dell's Chromebooks are a success. They're shipped in large quantities. They have no driver issues. They run coreboot. So my advice is simple: if you're a company trying to sell to folks who want a "Linux laptop", then stop sourcing components/machines from otherwise clueless Wintel vendors and trying to shoehorn a traditional distro onto that. Start with the BOM used in Chromebooks. Approach Dell and say you'd like to place a large order—because that's the business they're in, by the way. Now build your company's products on that platform instead.
Substitute "Dell" there with any other competent vendor that has demonstrated their hardware plays well with the FOSS stack powering ChromeOS .
> Partnering with Google for the chromebook supplychain seems like the better theory/plan.
No, that's an ignorant theory. Google is not known for hardware, and certainly not as a supplier for commodity hardware. Even ignoring all that, the response you should expect from Google in regards to playing along should not be enthusiastic agreement.
Can you please consider offering ECC memory in your laptops?
ECC is a very important technology for businesses. Errors in non-ECC memory, while somewhat rare, are not once-in-lifetime-of-universe, they are more like 1 bitflip per year kind of thing, which for some businesses can be considered worth talking about.
There is a demand for this, and almost no one is doing this.
The only other company that has done it so far is Lenovo, and if you know anything about privacy and hidden adware and history of spyware - you will not buy any Lenovo product.
ECC memory would require a Xeon, but it would be possible to underclock it to bring it to a mobile heat emission profile.
I've got a T470P (i7-7700hq), that's a 45W part in a 14" laptop.
If I balls to the wall it I can kill the battery in 90 minutes (though they are swappable) but I average ~6 hours under actual work conditions (VMs and Intelligent).
It spends the vast majority of time sat at a few percent utilisation but the processing power is there for those few second operations you run infrequently (build from scratch, indexing a large project etc).
I love it frankly, not turned desktop on at home more than once or twice since July.
Do you expect any flak from Intel on this decision? I hope not, because I'm going to be in the market for a new laptop in the next year or so and this just put you guys at the top of the list.
To compile to the same bit pattern binary, you'd need the same optimization settings (easy) with the same compiler (harder).
Intel can just give the source, but you can't trust Intel to just give you the same compiler they used, because the compiler might insert a backdoor. You'd need the compiler source, audit it for backdoors, then compile that with a trustworthy compiler. Then use the resulting compiler to compile the ME source.
The comparison between a person administering an unsecured computer network and a drunk driver has just made my list of legal IT analogies, along side BitTorrent being a car that might be used as the getaway vehicle in a robbery.
> You'd need the compiler source, audit it for backdoors, then compile that with a trustworthy compiler. Then use the resulting compiler to compile the ME source.
It is possible (though somewhat time-intensive) to audit binaries, too. If there is real demand for this, it should be possible to crowdscale the auditing problem over a large group of OSS enthusiasts.
Auditing binaries wouldn't really do anything as it's their hardware that'd run the binary. So the hardware can be programmed to lie or to still have some backdoor.
As long as the build is reproducible then they don't have to open it up. It would mean however that you'd still need to audit the compiled program to find anything injected in by the compiler.
Reading this:
> System76 will investigate producing a distro-agnostic command line firmware install tool. Follow us on your preferred social network for updates.
I am wondering. Have you considered using the [Linux Vendor Firmware Service](https://fwupd.org/) ?
It doesn't look like any work at all is actually being done on fwupd from my point of view. I've basically given up hope at this point that System76 is going to do the right thing.
To answer more harshly than before, you did not really try to help us get into LVFS. Instead, you continued to question our method of updating firmware, even though we have open sourced a large amount of our infrastructure.
Still, I have been changing our firmware updater to better suit LVFS and fwupd for when you finally decide to reconnect in a more positive way.
Seems a little unfair to announce a general indictment of System76 like that, to this broad audience, based on conversations only you and they have seen. Why make the account and escalate the drama? I'm sure both sides are doing good work.
Not trying to tone police you, but you run a project that necessitates bringing vendors to the table. Odd to publicly bash one that's trying to work with you.
I am working on embedding all firmware flashing files into one EFI executable when it is built by our CI - this should make it much easier to use from fwupd.
Hopefully you and the LVFS,fwupd maintainer both cool down a bit and get back to the discussion table.
You will both make history here, with more laptop sales and more awareness of LVFS+fwupd. Win-win, as I see it.
All the best.
I have so much respect and a little envy for the work you're doing at the moment. You're working on one of the most far-reaching and undeniably positive software projects I've heard of in years. Best of luck!
It's possible that at some point (depending on if it gets open sourced) I'll be grabbing some of your work and trying to wrangle it to work on my desktop. Is this an area where I seriously risk bricking my CPU, or are there safeguards that cause these kinds of instructions to either work or safely fail?
Yes, you risk bricking your computer. You can create a Raspberry Pi with a SPI clip for fixing bricked computers.
If you have BootGuard, you will brick your machine, unless you have the manufacturer signing key. We have the signing key for our laptops, so we are able to sign updates for the four systems with BootGuard: galp2, galp3, lemu7, and lemu8.
> Is this an area where I seriously risk bricking my CPU
As far as I'm aware, there's no particular state involved for you to brick the CPU, but you can easily write a firmware which will prevent boot, or fail some time after boot (In the case of removing critical ME blobs). You could brick your board temporarily, but you're going to be using an external flasher (with a convenient little SOIC clip :- ) anyway, so fixing it is a matter of writing back the old/working image.
This is, of course, assuming you are able to write anything that works to the flash, which you might not be able to thanks to BootGuard.
Does Ubunutu need to remain on the laptop after the driver has been loaded?
I'm aware that you're working on a distro-agnostic tool. In the interim period, however, if someone with a System76 laptop wants to use Fedora while still disabling ME, can they simply load stock Ubuntu/pop_OS long enough to install the driver and disable ME and then wipe the drive and reload their OS of choice?
Firstly, the ME section is locked until a message is sent to the EC. Flashing firmware on a live system can be dangerous, more so on a laptop. Our flashing method has to take place in EFI.
Signing keys are fused into the CPU for boot guard when the CPU is attached to the motherboard during production, for soldered CPUs like the U and H class. Having a customer signing key would seriously complicate BIOS updates, as only one key can be utilized, meaning our firmware updates would not work.
It would also make returns much more difficult, as all CPUs with customer keys would have to be replaced.
Right now, boot guard is only used on the Galago and Lemur.
There's no way to use TPM as a store for these signing keys because they are needed too early in the process?
Yes, I expect as a general solution for your customers it might be overkill, but as this is basically the cutting edge of public domain research on these topics it would nice to know what's possible for customers who would take the risks.
Understood, I have more reading to do in to how the EFI shim/flasher works.
Give us a System76 laptop with a TrackPoint and you will need to start carrying a snorkel to be able to breathe underneath the piles of money I will throw at you.
(It's never going to happen, I know. But a fellow can dream!)
This - coming from thinkpads the trackpad / keyboard appears to be too much of a downgrade for me to spend money on a System76 but I have been following them for some time.
Was it damaged at all? I would suggest sending it back. In the future, don't accept packages from UPS that show obvious signs of mistreatment. Although I'm not really sure how to do this if they just leave it on your front porch.
On the side note - a new initial setup was released today for Pop that may have fixed your account creation issue. I am sorry if you ran into problems with Pop.
Good to know. I haven't decided if I'd prefer going vanilla Ubuntu. I'm going to give Pop some more time yet. I can see benefits; I'm just not sure they're for me.
I'm not too familiar with the inner workings of Intel ME. Have you verified that setting the HAP bit is effective with the ME code removed? If so, then that is obviously the way to go. However, if the code that interprets the HAP bit and shuts down the ME is removed (as well as everything else not necessary for the computer to run), which option is actually safer, HAP or code removal?
The safest option is to do both, and that's what's happening here (though the post is not clear on it, they're running me_cleaner with options that both erase all but the initial loader module of ME, and set the bit so it disables itself after startup instead of loading other modules)
I think gnu8 was asking: Does removing the code interfere with HAP mode (e.g., by removing code necessary for HAP to be activated, maintained, operated, etc.)? Is both really safer than one or the other?
For example, there may be two drugs that can treat my illness; taking both might not be better than taking one or the other.
Would be a good place to start, as far as not supporting dell goes. Linux is also a first class citizen w/ system76 - Dell has the factory linux xps13 e.g. but I've heard from a handful of friends who use one at work that its obvious Linux was just kinda slapped onto the laptop at the end of manufacturing or whatever. Support is also probably much better with System76.
My home laptop is from a Kudu Pro from System76, while my work laptop is a Dell Precision 5520, with Ubuntu installed. My current Kudu is a replacement for another Kudu that died after a number of years.
For my particular case, the Dell has superior hardware (also, it's about a year newer), but its Ubuntu install has strange problems, and I get crash notification popups somewhat often (including immediately upon logging in for the very first time). The Kudu's install has been rock solid.
I work on Redox at night and on weekends. I try to write software that can be used in both places.
For example, the graphics library orbclient was used in the firmware updater, and the UEFI library from the firmware updater may be used for a Redox UEFI bootloader.
The article and repo seem to be targeting Laptops and there seems to be some specific desktop-related lines in there. In the laptop side of things, it clearly states the Intel ME does not have any function, but it does not say the same for desktops. I imagine the ME is also useless for desktops, but maybe I'm wrong here and there is a reason this distinction is made?
Why why why won't you just use LVFS? A number of OEMs have absolutely no problem delivering updates through all sorts of tools and channels.
It feels NIH and is exactly why I'll be buying Dell and never System76. I can one-click flash my Dell forward without configuring or installing anything.
For that matter why is there a "HiDPI Service" running? Why is the driver a set of udev rules that could've been upstreamed?
You don't need to buy branded laptops to be able to disable Intel ME. Thanks to me_cleaner, I have been running my Chromebooks with Intel ME disabled for years. You can do it too if you have 30 minutes and a raspberry pi.
What's even more interesting is that there is a simple, automatic, no-frills method to clamp a SOIC clip connected to a Raspberry Pi zero to any Chromebook and it will clear out the Intel ME automatically with no user interaction. It's not even hard, all you have to do is to configure the Pi to enable SPI, and then make the pi automatically run flashrom to pull out the ROM from the Chromebook's flash chip, run ./me_cleaner on the ROM image, and then flash it back.
It can be done safely and automatically, and you wouldn't have to risk frying your laptop, so everyone could do it provided they can open their laptop. However, I'm too lazy to document it properly: either by providing the image tool to create the said Raspberry Pi Zero image + h/w instruction or by providing a premade hardware to do it.
I have reasons to believe that downloading a random binary image from a random guy nicknamed jimmies and use it to flash the firmware to your laptop because you don't trust Intel is probably not a great idea. The act of creating such a script to customize Raspbian, and testing out to make sure that it works for every Chromebook or laptop, and make a hardware compatibility list is quite a daunting task. I was talking about that briefly to some of the security people, but then as a grad student trying to graduate it got to the pile of TODOs. So if anyone is interested in it, let me know and I can provide some more details.
Currently, I'm running a Dell 13 inches Chromebook that can be had for $300 and does everything I need.
While this is great, and I'm glad it exists, not everyone is like you and I: willing and able to open a computer and use a SOIC clip and a Raspberry Pi to purge the ME from the computer's firmware. You and I are what the normals tend to call geeks or nerds, and from my experience amongst geeks and nerds, one of the less common subtypes. That's a depressing way to look at it, and one could argue that it's a condescendingly elitist view, but that doesn't change the truth of it. I'm confident most other posters here could immediately think of a half-dozen friends and family members for whom even opening the computer was deep magic, nevermind going after the ME with a SOIC clip.
For the rest of the population, there's value in there being a vendor who sells magic black boxes, certified to be freshly purged of all known evil demons.
>For the rest of the population, there's value in there being a vendor who sells magic black boxes, certified to be freshly purged of all known evil demons.
Sure, but there are things that I like about my Chromebook that the System 76 laptops, despite being way more expensive, are unable to to provide. I like how mine has a backlit keyboard, has a FullHD 1080p IPS screen, and an 10 hours battery life, all that for $300.
I just looked at the number of people who upvoted my post and I am really think that this actually could work if there is a person who builds the hardware, which I think is trivial. The problem is that not many are comfortable with both the electronics (wiring the clip) and be able to compile a big software package with dependencies on the Raspberry Pi. Make the frustrating 2 hours job to a 5 minutes job of opening the laptop and identify the chip, I think it might just be what many people want.
I talked to a very smart netsec person and he just said that because he doesn't understand electronics, he couldn't have done it himself, but if that's something prebuilt or foolproof, he would absolutely do it.
Perhaps I will actually take the plunge and publish the work over the next couple of months as a Show HN post.
If it became popular, then there would be a cottage industry in doing it for people. Charge them say $20 for the labour of opening the thing up, running the tool, reassembling and testing it.
This already exists for "chipping" games consoles and the like. I think it's just that the average person can see the cash value to them of pirating games while the ME has no effect on their daily life.
This sounds like it makes sense to offer as a service: You should know how to protect against electrostatic shocks and preparing everything is time consuming. It scales much better after the first time.
Maybe mobile phone repair shops would be interested in offering?
This is something I would love to see. As evidenced by the netsec person you spoke to, not everyone's comfortable with electronics. But a pre-built thing or kit that leaves little to no chance of user error would provide a great deal of value to a great many people.
> I like how mine has a backlit keyboard, has a FullHD 1080p IPS screen, and an 10 hours battery life, all that for $300
That sounds very compelling. Which model are you using please? I'd like to take a closer look but I don't seem to see one with these features/price - but I am in the UK.
The Dell Chromebook 7310. They don't sell it anymore now. They used to have the option to choose Celeron/i3/5/7 and 4/8GB of RAM. I have the 8GB i3 one.
Yeah, I pretty much just recently learned enough electronics to feel comfortable doing something like that and I still haven't done it on my hardware for fear of bricking it.
I'm just hoping for a 100% software exploit that will do this.
You're making a mistake to make it sound so casual, at least put in a warning to always backup the bios image first before flashing the chip. Not all boards are the same, for example I've got a gigabyte board,I've flashed the bios just setting the HAP bit and it bricked the board because that part of the image segment is checksum validated. Luckly I had the backup ready to flash back so there was no downtime.
Can you define "everything you need?" Cause I was very lucky and won a chromebook (can't remember exact model off the top of my head) and tried to use it as a programming tool (did the "Make Chromebook Run Ubuntu" thing) but it was so slow so as to be completely unusable.
Yup, I have looked at the procedure a few weeks ago and found it really easy to manage and ordered the hardware (a pomona clip in my case) to do it. People tend to be scared about opening up their computers, but this is really not hard at all and since you keep a copy of the ROM image, you can always fix it later on should you brick anything.
> System76 will automatically deliver updated firmware with a disabled ME
Having the ability to automatically push new firmware of your own creation to customers' machines is more power than you ought to want. My security threat model as a System76 customer shouldn't have to include you (perhaps with you being hacked or coerced) pushing me malware that's undetectable to my OS after it's been installed.
Please reconsider this feature (of automatic firmware updates). Firmware updates are rare enough that it should be fine for them to be explicitly opt-in. It's great to want to make Intel's firmware more secure. But replacing Intel as a possible attack vector with yourselves is strictly worse for the machine's security.
System76 employee here. I suppose the "automatic" could be re-worded; the data is automatically pushed to the machine but then the user always opts into the actual install.
But on the other hand, System76 customers are trusting that System76 hasn't been hacked or coerced to ship malicious firmware from the factory in the first place. These updates are signed and verified with industry best practices.
jackpot51 (the System76 engineer currently working on this) could probably detail it better than I can, though.
This sounds pretty good. But just to finish exploring this:
> But on the other hand, System76 customers are trusting that System76 hasn't been hacked or coerced to ship malicious firmware from the factory in the first place
I think it's reasonable to feel differently about those two risks.
Most notably, you only get one chance to load malware at the factory, whereas you have an infinite number of chances to push malware as a software update after that. It's harder for you to avoid being compromised forever than to avoid it at one specific moment. One person on your team could probably get malware signed and distributed as an targeted update without anyone else knowing, whereas doing it in the factory might take more coordination.
It's also tidier from an attacker's perspective to deliver malware just-in-time to a specific user, rather than to everyone, or to a machine that you hope will end up in the hands of the target weeks/months later. It's less detectable.
If you have a way to avoid being able to infer (e.g. by their IP address, correlated with other records) which human is asking for a firmware update file (or any update file) at the time it's downloaded, I recommend taking steps to deny yourself that knowledge.
Is the user intervention required to install the firmware or do you still have the capability to install it without asking? Just because you ask for permission now does not mean that you have to.
I asked David Jordan, the engineer working on the firmware updater.
>Not only aren't updates initiated without permission, I wrote the code to make that literally impossible without changes to the installed python code.
Thanks for the reply! So you can push an update to the python code that allows you to push an update to the firmware without prompting? Sounds like we still rely on the security of your systems to prevent malicious firmware from being pushed.
Yes they have to be able to update the system but in this case they are also able to update the firmware without asking which means anyone who can impersonate or coerce them can also update the firmware.
Since the firmware updater is a Python program, you can audit the source code by looking at the relevant directory in site-packages before you accept. If you're really paranoid you can set up a periodic script that sends you an email if the contents of that directory change.
It is delivered through the PPA that you enable after trusting system76. So I don't know where this reasoning is comes from. Also updates in Ubuntu are delivered only after the user explicitly invokes it(in GUI or using apt).
Afaik the standard setup of Debian and Ubuntu is to install UnattendedUpgrades with automatic periodic installation of security updates. These aren't supposed to come with functional changes beyond patching security bugs, but there's no theoretical reason a security-patch couldn't bootstrap netbsd and change the entire os. It's code that runs as root.
To be clear: I think this is a great default configuration, and it's been a long time coming - but it's just not true afaik that Debian/Ubuntu won't auto-update at all in the default configuration.
"As of Debian 9 (Stretch) both the unattended-upgrades and apt-listchanges packages are installed by default and upgrades are enabled with the GNOME desktop."]
This is a strong contrast with my experience in trying to patch the vulnerability on an Asus desktop motherboard.
The process was so byzantine that I very much doubt more than a small fraction of home users would get through it, or even bother starting.
The correct steps were (1) flash a newer bios, (2) install the Intel ME driver for windows, (3) run the actual vulnerability patching tool. Discovering those steps required a bunch of trial and error and navigating Asus's really terrible website full of badly named downloads.
> This is a strong contrast with my experience in trying to patch the vulnerability on an Asus desktop motherboard.
This is why I don't buy "enterprise users" as a reason for having IME. I've never once worked in a company that patched firmware, even though they have specialists capable of it. They want the option to perform enterprise wide upgrades with ease but they aren't willing to pay the true cost of having this ability.
> Discovering those steps required a bunch of trial and error and navigating Asus's really terrible website full of badly named downloads.
Gets even worse when your international. Half the links will be to a US address that will then try to redirect you to a localized one which will then not have the resource you were looking for. Then you've got some really byzantine export restriction procedures and you have to create an account but it still probably won't work. I've had these issues with ordinary drivers too, it's the biggest reason I support the linux in kernel tree and no stable ABI model, it's better for users.
Removing ME [almost] entirely by editing the official bios image with me_cleaner.py and then updating the bios the normal way sounds like it is simpler, if you don't need the extra network-connected ME features there.
I'll admit I was a little afraid of bricking the PC, as with any kind of BIOS modification, but it worked like a charm.
This is awesome. System76 does a lot for the Linux community. Everyone reaps the benefits of their work, even people that aren't customers. One example from just this blog post:
> System76 will investigate producing a distro-agnostic command line firmware install tool. Follow us on your preferred social network for updates.
They sell re-branded clevos with marginal build quality. Some models have been unusably defecitve. I wish they were the Linux-laptop company that many hackers dream about, but they just aren't.
There was another comment on another threat that said that, although they do buy Clevos, the cheaper ones Clevos sells aren't up to the same specs. They also order really specific hardware for their branded laptops that have known working drivers. Either that or they work on getting the correct binary blobs working and merged into the official linux-firmware package.
Things like getting newer drivers working (and contributing that openly back to the community) plus getting ME disabled are some pretty big value added. If they had a 4k model I might be more interested.
System76 employee here! We do have a number of HiDPI models.
Galago Pro (https://system76.com/laptops/galago) is probably our most popular, and its resolution is 3200×1800—not technically 4K, but the right resolution for that small of display.
Oryx Pro usually has a 4K HiDPI option, but it looks like it's currently sold out. Our WS models also have 4K options, but they're pretty specialized machines.
Things may have changed in the last year or so, but plenty of System 76 Clevos have been notorious for problems in the past. I'd be interested in knowing that things have improved significantly because I'll probably be upgrading to a new laptop around the middle of next year.
For me probably the biggest issue is that they usually have mediocre keyboards and sometimes they even have keyboards that drop keystrokes. It seems like every model they release should have a top-notch keyboard because their target market is mostly developers.
Me too. I'm running the Dell Precision 5520 right now and I've been pretty happy. Tho, deviating from the stock OS (Ubuntu 16.04) is not a very easy path. For that reason I may go back to System76 for my next buy.
I have a 5520 too, but one I got via the outlet store, and which came with Windows 10, not Ubuntu. I've since partitioned it and have Arch running on it too. Was completely painless, and everything works. Which distro are you having issues with, and what are the issues?
> You must run Ubuntu 16.04 LTS, Ubuntu 17.04, Ubuntu 17.10, Pop!_OS 17.10, or an Ubuntu derivative and have the System76 driver installed to receive the latest firmware and disabled ME on laptops*
Ubuntu is the distribution that once sent everything you typed into the desktop search box to Amazon so that it could deliver you ads. Current versions may not do that but it's clear that Canonical prioritizes profit over privacy.
It's disappointing that if you choose not to run Ubuntu you can't take advantage of their firmware update tool.
If we don't criticize people for doing bad things they should never have done, how can we expect other people to not do those bad things in the first place?
It was always something people could turn off, and they were very open about the feature existing. This wasn't the cross-site ad tracking that's probably more severe on every website you go to.
If it was something that was off by default you could turn on that would be fine, but the fact that you have to turn it off implies that they are taking advantage of people who aren't aware of the feature's existance in the first place.
It’s the way the wind is blowing. Even Firefox is infested with invasive, on-by-default telemetry now. The price of freedom from it is eternal vigilance. I’d very much prefer Canonical not to get it into their heads that it’s OK to do it again.
This post is about System76 delivering firmware for System76 machines, which are shipped with either Ubuntu or Pop!_OS. I'm not really disappointed that a hardware manufacturer is releasing their firmware update tools for their supported OSes first. (If you are not running their hardware, this firmware update tool is presumably not for you, anyway.)
Yea I'm sure we'll see a more generic command. This was probably just to get things out the door, quickly, with the default operation systems they have installed.
Keep in mind, testing this type of update is pretty important. I'm sure they went through this several times with multiple versions of all their hardware.
Honestly for the sake of not bricking a laptop, I think they should release a bootable USB image for other Linux distros and operating systems. Flash it to a USB stick and now you have a completely stable environment, with the kernel you've tested against, and remove potential unknowns.
There could also be issues with dependency versions of the executables that prevent cross-distro usage; it’s difficult to run the same executable on e.g. Ubuntu and Fedora unless it only depends on glibc. Even then you have to worry about glibc symbol versioning.
> Even then you have to worry about glibc symbol versioning.
Symbol versioning will actually permit the binary compat (unless you have a rather clueless approach), and not break it by some kind of "oh my god I don't know what is this black magic and I would rather not be bothered to learn anything about it even if it is my job to release binaries people can run, so let's just forever rant a little about symbol versioning and pretend GNU systems are hard to target".
Now in order to release a binary which will work on multiple systems, obviously you can't rely on symbol versions that are only present on some of those systems but not all.
If you intent to at least test it on those targets (and well, you... should? ) then obviously you have access to them and obtaining a toolchain on the "oldest" one is trivially an apt-get (or equivalent) away.
Basically, if you want the simplest worryless solution that will work 99.9999% of the time: you build your binary there. Done.
You might even be able to find some compilation flag or something like that if you insist on building on a recent distro, while still targeting old ones. I have not checked; I would if I had to do that.
I was unable to find such a flag last time I had this issue. It’s not black magic, but most other platforms don’t require you to build on the oldest platform you support. That can be quite a pain if that version hasn’t backported newer versions of the tool chain you are using, requiring you to do a custom build of that instead of using the package manager. So not quite an apt-get away.
My ~2011 motherboards BIOS lets you reflash it from INSIDE the BIOS, from a connected readable storage device, i.e. a flash drive. It eliminates any need for OS-specific update tools. I am surprised this isn't more common.
This simply does not scale. Every single customer has to download the BIOS, usually from a questionable source, plop it on an unreliable flash drive, and then boot into the setup menu to manually select the right file. Our method will automate most of this process more reliably.
Firefox does the exact same thing with Google and other search providers and gets significant revenue, and nobody accuses them of 'putting profit over privacy'
Bit hypocritical to pan Ubuntu while giving Firefox a pass. At least for Ubuntu there are many alternatives.
This was probably a misguided attempt by Canonical to generate some revenue, they recently had to lay off a significant number of employees. Criticism is valid but dismissing the entire Ubuntu project because of a single bad decision taken and reversed years ago is uncharitable and over the top.
Every single thing you type in the search bar is sent to Google or whoever they have done the search deal.
Ubuntu used to do the exact same thing with Amazon, its an affiliate link and does not 'display ads', that is complete FUD. Please understand the issues before commenting.
It's a search scope for amazon so you can get amazon search results in the unity search bar if desired. Firefox has also done a privacy busting ad deal with Cliqz where user's browsing history is sent to cliqz servers for targeted advertising. This is even worse for user privacy.
This isn't a software update, it's a firmware update. I don't expect updating firmware to be painless; and I'd rather it not be painless. Updating a PROM is a messy process fraught with peril, I want as many checks & balances between my clumsy fingers and that flash as possible.
> With whatever else a normal human has going on in life? There's no way.
Many vendors don't ship ELF binaries update their firmware at all, and yet I still manage to flash their devices. I've lost count of how many times I have to throw FreeDOS a USB drive to flash the RAID controllers I use. Putting Ubuntu on a flash drive to install a firmware update is hardly any more complicated than that. (At a minimum: at least you'll have a reasonably competent shell and functional networking w/ an Ubuntu live image.)
Think about what you are asking first. A VM is a virtual machine that theoretically is separated from the base hardware. I don't think a VM is the solution for Bios Updates or changing bits on the physical hardware itself.
Of course, you can monkeypatch the VM's bios all you want! But that would be erased the next time you start a new vm.
The only other way is to find a security hole in KVM, qEMU, or XEN, and then exploit it to break out of the VM[1] and get access to base hardware. Hard to do, but it does happen.
This would be my way of handling it. I usually keep a usb stick around with ubuntu on it just because the live version of it is incredible useful as a tool.
Hi I'm a mostly average Linux user just now learning about these hardware back doors as I'm planning on building a new computer. Can I avoid the disadvantages of ME et al by building my own computer from a separate motherboard and CPU? I remember reading that somewhere but I haven't see it explored again.
My first thought was "Ah ha! AMD CPU!", but they seem to be in on it too. What's a "normal person" to do?
* It might be worth looking into: ARM, Intel Atom, or POWER9 processors
* Intel chips without VPro seem to omit much of the ME functionality, including AMT which provides (all of?) the remote access capability. AFAICT, chips with the Small Business Technology implementation of ME are purposely designed to not have remote access. Maybe use one of those.
I've heard a lot of great things about them and I love their mission statement, but I've also heard a lot of bad things about availability, shipping, and spotty customer service (but again, occasionally great customer service). I'm not sure what to think and whether to get my next laptop from them.
It also makes me nervous that they're putting resources into phones when they obviously haven't solved all the issues with non-free components of laptops. It gives me Firefox OS and mobile Ubuntu vibes.
> It also makes me nervous that they're putting resources into phones when they obviously haven't solved all the issues with non-free components of laptops.
By not using x86 (the board is going to use i.MX8), they remove a whole series of issues that plague modern laptops (no ME, no EMC, etc). They have a very long run-down on why they chose i.MX8, and the freedom status of the components[1].
I'm thinking of buying a Purism laptop. The only thing stopping me is that I just bought a ThinkPad a few months ago and it feels wasteful to buy another laptop so soon...
You can get the same laptop from a different distributor for a lower price. Try searching for 'Clevo' or 'Sager' laptops. Clevo is the company that makes the base model, but they don't do any sales to consumers. Sager is another name under which the same laptops are typically sold.
They announced earlier this year that they were going to start doing their hardware design in-house[1]. I doubt this will make them any cheaper but might help on other fronts at least.
Except that they test and fix hardware compatibility issues for linux, do driver development and they push it upstream.
Their hardware is usually jointly developed with safer/clevo, which sager is free to sell barebones
If system76 goes out, you can be pretty sure that sager Linux compatibility goes out with them (along with many other brands that use the same wifi chips, etc)
I've got a Lemur from System76 that I've had for several years, program on it almost every day, it still works well. You can probably get something cheaper, but it's not like they're Apple-expensive, and if there's a bit of $$ going towards efforts like this, I'm glad to support it.
Well, for $1000 I can get a Surface Book 1 and run Ubuntu on it, and it's absolutely perfect.
The Galago Pro is almost something I'd get, but I really dislike actual buttons on the trackpad: most of them are (rightfully) clickable anywhere on the trackpad.
> Well, for $1000 I can get a Surface Book 1 and run Ubuntu on it, and it's absolutely perfect.
Are you referring to installing Ubuntu on the WSL, or actually wiping out Win10 and installing Ubuntu natively?
Because to my knowledge, Linux support on the Surface/Pro/Book line is still wobbly at best. I've seen some people running Arch, but problems with battery life and sleep mode abound. I can't imagine the Surface Book's detachable base in tablet mode would be working yet.
I pieced together how to update a ThinkPad's ME firmware without Windows last week: https://news.ycombinator.com/item?id=15744152 - the process is a pain, as Lenovo only provides a Windows binary for installing the update, so you have to piece together a WinPE image that contains the ME driver and firmware update tool, boot it, load the driver, and then execute the firmware update.
The process is probably the same for most other vendors, so if your vendor provides a Windows ME firmware updater (no pun intended :P) you might be able to make it work that way.
I used me_cleaner on my old as heck gigabyte sandybridge desktop and was surprised how straight-forward it was. I ran the python script against latest rom, reflashed, everything works and I confirmed it's disabled.
Great to hear that you kill ME but I'll still stick with puri.sm or alpha.store because systen76 machines are just too freakin overpriced. They talk about freedom and FOSS yet their machines can cost 1.5 times a macbook, with free OS and rebranded chineese 200$ clevo cases. Available price is an essential part of FOSS and OSHW phylosophy, greed never had any place in this community
I wonder if system76 might consider laptops with alternative architectures like POWER9. A number of them are open all the way down to and including their concept of microcode.
Is there any appliance or software that can passively monitor a network to see if any ME commands and related data flow back and forth from an attacker?
It seems to me that using ME as an _actual_ backdoor would be only an occasional thing, (maybe once in a lifetime thing), but it would be so cool to at least know when it is happening and maybe capture some packets.
You mean AMD PSP. And I don't think it has received the same attention as ME, even though it's more or less the same backdoor. Which they minitruthfuly advertise as For Your Own Safety:
It happened more than a year ago[1]. It doesn't remove ME entirely, but drastically reduces the attack surface in the currently accessible ring - the GitHub project has more details. Ultimately, the network-enabled bits that keep getting exploited are removed.
I can't buy System76 laptops, because they only have US keyboards. UK and Scandinavian (the one I'm using) have more buttons, so I can't just fix it with stickers.
How so? You're still buying a system with an Intel CPU. Intel still gets their "dues". The number of people that care about this is a fraction of 1 percent of their user base.
Just so you don't misunderstand me, I am one of those people, and what Purism and System76 are doing is great.
I just don't think it's going to affect Intel in any way whatsoever.
It probably won't but if it shows an uptick for System76's business then you might see the other companies who actively ship systems that devs pay attention to doing the same. If the action ever gets Dell or Lenovo to follow, it's still progress.
Relevant source code can be found in the following places, keep in mind that it is still work in progress:
- System76 Driver with Firmware Update support: https://github.com/pop-os/system76-driver/tree/firmware_artf...
- Firmware Update Frontend: https://github.com/system76/firmware-update
Please ask me anything