China Embedded Spy Chips On Supermicro Motherboards

I watched the report and read the entire reporting.
Tbh, without actual proof I just don't buy it -- especially due to this statement: "companies do not vet the hardware they use".
That is complete and utter garbage and pretty much shows this guy doesn't know WTF he is talking about.

Without conclusive evidence, this smells like market manipulation on the coattails of China-fearmongering and pure bullshit to me.
I don't trust Chinese hardware (ie. Huewei etc), but I do still trust US businesses to scrutinize the hardware they purchase and integrate.
It was very well hidden though right? Although it was eventually found.
Maybe some companies do vet it and others don't, and he's saying the reality is not enough do.

All this will have to be taken care of, there has been a few stories about Military Jets compromised but caught before the parts were used, and at this point the people in power will need to up their game to catch anything like this or move it to US. Even still the world is so connected, because we couldn't make every component even if we wanted. Glad I'm not having to decide on this stuff, it's a mess.
 
How is this a tech enthusiast forum?
Have none of you worked in a colo, built out racks, deployed environments?

You actually think that that some embedded hardware has co-opted a hypervisor to send traffic out?

It's credible to you guys that no one in multiple NOCs noticed transmission to an unknown IP? You think that it's possible something would just mirror traffic and no one would notice?

Do you guys not maintain your hosts, network gear, storage, and most importantly monitor your bandwidth consumption so you can balance your clusters? My NAs are constantly updating their gear bc SDN saturation is miles away from watching your Cisco 1Gig hardware.

A single embedded super custom chip is Xen aware, Openstack aware, whatever RH based stuff Apple is using, VMware aware? Teensy little custom chip manages to send rack after racks worth of data bc it's aware of rest states where it can dial home when the line(s) it's connected to aren't saturated?

Wow, every node talks to each other and can superset itself bc it knows what rack config it's installed in.

I've built Rhel, Centos, Xen, VMware, Azure, and AWS environments.

Please tell me about this wonder grain of rice can write a new zone into my hardware security modules without me getting an alarm that a crypto-user has committed a write and somehow got the required 2nd person to push it.

Bc this ain't your janky home lab or barely competent startup full of glorified front end devs we are talking about getting owned.

I have node and react Devs laughing at the idea of this, and they know about jack nothing about running physical or code defined infrastructue.
 
Given the statements I would rather believe Amazon and Apple's. They went on the record refuting all parts of these claims. You bet that if they were not 100% sure they would at least leave some wriggle room in their statements. If it is found that Amazon and Apple lied, they would be facing HUGE fines from the SEC as well as from individual investors. For comparison just look at what a tweet cost Musk, and that is not even an official statement from Tesla. Putting out intentionally wrong information is a big big deal and people can go to jail over it. On the other hand, Bloomberg can always hide behind its sources and journalistic privilege.
 
  • Like
Reactions: Meeho
like this
What Bloomberg printed was a Verge level stupid PC building vid, but apparently people think colo gear is plugged into a switch with DHCP turned on.

The legions of business accountants that calculate fractional use of resources to bill customers just totally missed the giant leak to China.

Not like any Sr NA isn't spending the majority of their time on bandwidth the same way I'm spending my time on compute and storage.

We just totally wouldn't notice any anomalies that don't get forwarded as a customer billing item.

I mean I've listened to some straight up idiots today propose that they write tools in 3 months time that somehow Apple, Amazon, NetApp, Cisco, Juniper, Linux Foundation, Docker, Google etc etc etc haven't written based on this article.

You solve this on your own, who needs the Cilium project?
 
How is this a tech enthusiast forum?
Have none of you worked in a colo, built out racks, deployed environments?

You actually think that that some embedded hardware has co-opted a hypervisor to send traffic out?

It's credible to you guys that no one in multiple NOCs noticed transmission to an unknown IP? You think that it's possible something would just mirror traffic and no one would notice?

Do you guys not maintain your hosts, network gear, storage, and most importantly monitor your bandwidth consumption so you can balance your clusters? My NAs are constantly updating their gear bc SDN saturation is miles away from watching your Cisco 1Gig hardware.

A single embedded super custom chip is Xen aware, Openstack aware, whatever RH based stuff Apple is using, VMware aware? Teensy little custom chip manages to send rack after racks worth of data bc it's aware of rest states where it can dial home when the line(s) it's connected to aren't saturated?

Wow, every node talks to each other and can superset itself bc it knows what rack config it's installed in.

I've built Rhel, Centos, Xen, VMware, Azure, and AWS environments.

Please tell me about this wonder grain of rice can write a new zone into my hardware security modules without me getting an alarm that a crypto-user has committed a write and somehow got the required 2nd person to push it.

Bc this ain't your janky home lab or barely competent startup full of glorified front end devs we are talking about getting owned.

I have node and react Devs laughing at the idea of this, and they know about jack nothing about running physical or code defined infrastructue.

This entirely...

I've been trying to talk sense to people that believe this horrifically inaccurate article and they are all doomsaying. People with a ridiculous lack of understanding of the industry reading a bloomberg article claiming that Supermicro being "hacked" is going to cripple the data center world when SM only has around 5% market share overall.
 
Also, in terms of threats, Russia threatens our internal stabilization, Iran threatens Israel and our Middle East interests, yeah China competes with us in South East Asia, but honestly, I don't consider it as serious as Iran or Russia. Not enough for the label of enemy.
This is a serious error. China is the largest threat there is by far. No, they won't start a shooting war but what they will do is....
Take over and militarize islands that don't belong to them.
Claim seas and airspace that do not belong to them.
Put countries in 'debt traps' and take their land. Land they will very likely turn into military bases, there is evidence they've begun in some cases. Malaysia has been one of the few countries to tell China to piss off. To save myself a lot of typing, here's a link that has pretty much everything I want to say about the Chinese debt traps. You could also watch any number of videos on YT or read any number of articles from established news agencies.
https://www.quora.com/What-is-the-so-called-Chinas-debt-trap-policy-all-about
They interfere in government everywhere they go. Australia, NZ, Canada, the US, African nations, South American nations, the list goes on forever. At the same time, they claim to have a 'non-interference policy'.
https://www.foreignaffairs.com/articles/china/2018-03-09/how-china-interferes-australia
https://www.irishtimes.com/news/wor...very-level-of-society-report-claims-1.3516388
China is an imperialist nation.

Many of you guys have little inkling how nasty China is. I've read the most popular newspapers in China...in Chinese. They're disgusting. The level of propaganda and lies and misinformation and hatred would shock most westerners, it sure shocked me. They're published and approved by the government. I'm not saying western media is some paragon of virtue and integrity but you can find opposing viewpoints and information. You can't in China. Only one opinion is permitted and it's that of the CPC, only one source of information is permitted and it's that of the CPC.
 
Where the F am I going to find a made in the usa MB i can trust? Oh, nevermind - made in Taiwan, that forbidden word in china will work just fine.

And here are the china apologists right on time. What a joke.
 
How is this a tech enthusiast forum?
Have none of you worked in a colo, built out racks, deployed environments?

You actually think that that some embedded hardware has co-opted a hypervisor to send traffic out?

It's credible to you guys that no one in multiple NOCs noticed transmission to an unknown IP? You think that it's possible something would just mirror traffic and no one would notice?

Do you guys not maintain your hosts, network gear, storage, and most importantly monitor your bandwidth consumption so you can balance your clusters? My NAs are constantly updating their gear bc SDN saturation is miles away from watching your Cisco 1Gig hardware.

A single embedded super custom chip is Xen aware, Openstack aware, whatever RH based stuff Apple is using, VMware aware? Teensy little custom chip manages to send rack after racks worth of data bc it's aware of rest states where it can dial home when the line(s) it's connected to aren't saturated?

Wow, every node talks to each other and can superset itself bc it knows what rack config it's installed in.

I've built Rhel, Centos, Xen, VMware, Azure, and AWS environments.

Please tell me about this wonder grain of rice can write a new zone into my hardware security modules without me getting an alarm that a crypto-user has committed a write and somehow got the required 2nd person to push it.

Bc this ain't your janky home lab or barely competent startup full of glorified front end devs we are talking about getting owned.

I have node and react Devs laughing at the idea of this, and they know about jack nothing about running physical or code defined infrastructue.
Case closed guys, data breaches are impossible
 
A single embedded super custom chip is Xen aware, Openstack aware, whatever RH based stuff Apple is using, VMware aware? Teensy little custom chip manages to send rack after racks worth of data bc it's aware of rest states where it can dial home when the line(s) it's connected to aren't saturated?

Wow, every node talks to each other and can superset itself bc it knows what rack config it's installed in.

I've built Rhel, Centos, Xen, VMware, Azure, and AWS environments.

Please tell me about this wonder grain of rice can write a new zone into my hardware security modules without me getting an alarm that a crypto-user has committed a write and somehow got the required 2nd person to push it.
.
I can't speak to the rest of your post, but I think I can clarify this part.

The allegation is that the chip took control of the BMC. BMC is independent of the CPU, so BMC doesn't care what the CPU is running. BMC generally has usb access(keyboard, mouse, usb mass storage), video access, and power system access/fan access to the server. I dont know what the next step in the attack would be, but that's not really what the article is about anyways. What do you mean "custom chip manages to send rack after racks worth of data". The article doesn't mention sending large amounts of data.
 
Do you guys not maintain your hosts, network gear, storage, and most importantly monitor your bandwidth consumption so you can balance your clusters? My NAs are constantly updating their gear bc SDN saturation is miles away from watching your Cisco 1Gig hardware.

A single embedded super custom chip is Xen aware, Openstack aware, whatever RH based stuff Apple is using, VMware aware? Teensy little custom chip manages to send rack after racks worth of data bc it's aware of rest states where it can dial home when the line(s) it's connected to aren't saturated?

Ok couple things:

1) Do you really monitor bandwidth down to the level that you notice a few k/sec difference? Really? Come on, that isn't even feasible, much less useful. Well guess what? You don't need a lot of bandwidth for a covert channel, depending on what you are doing and what you are looking at. It isn't like someone messing with your systems has to be using tons of bandwidth. The channel can be very narrow, and it can be designed to carefully keep transmissions sparse.

2) You think your hypervisors are the lowest level of the system, that they are aware of everything that is going on? No not hardly. Your system has low level processes that go on that your OS is unaware of, unable to monitor, etc. System Management Mode, also sometimes nicknamed "Ring -2" is a good example. This is code that runs on the CPU itself, but the OS isn't aware of, running in memory the OS can't access. It is used to control shit like hardware that we don't want the OS to have to bother with... but if exploited it can do other shit as well. Here's Chris Domas demonstrating a real SMM rootkit. That's just stuff on the CPU itself. Other chips on the board can do things that the CPU itself is totally unaware of.

I'm not saying this is an accurate report, but this idea that you have complete visibility in to your system is bogus, and only grows more so as they get more complex.

Likewise while you can monitor all the traffic that goes through your network, presuming that gear isn't compromised of course, the idea that you can identify any anomalous traffic is silly. Unless your traffic is extremely small and extremely regular, there is going to be so much going on that the idea you can easily pick out bad traffic is laughable. If there were true, we'd have stopped all attacks over the Internet by now because ISPs would implement this magic tech. Really all we can do is pattern matching, behavioral analysis, and the like. So if something is well hidden, particularly if it is always there when you are doing baseline analysis, as a hardware compromise would be, it can be exceedingly difficult to detect.
 
"Because Apple didn’t, according to a U.S. official, provide government investigators with access to its facilities or the tampered hardware, the extent of the attack there remained outside their view."

"In one case, the malicious chips were thin enough that they’d been embedded between the layers of fiberglass onto which the other components were attached, according to one person who saw pictures of the chips."

!!!
 
and this is why you tightly firewall your out of band management on any server... this is why dedicated management vlans exist and why any remote user should be VPNing into such a network with plenty of security along with it...
 
  • Like
Reactions: mikeo
like this
This is even scarier though. Supermicro doesn't design its products in China to my knowledge. These spy-chips were surreptitiously inserted during the manufacturing process.

This puts not just Chinese brands under suspicion, but every single electronic device manufactured in China, which is essentially all of them.
Its true, finally some people are wising up to the fact that china's "cheaper goods" may not be the better if I means they steal trade secrets and government plans.
 
I can't speak to the rest of your post, but I think I can clarify this part.

The allegation is that the chip took control of the BMC. BMC is independent of the CPU, so BMC doesn't care what the CPU is running. BMC generally has usb access(keyboard, mouse, usb mass storage), video access, and power system access/fan access to the server. I dont know what the next step in the attack would be, but that's not really what the article is about anyways. What do you mean "custom chip manages to send rack after racks worth of data". The article doesn't mention sending large amounts of data.

Yeah, I don't get why everybody thinks an attack would be one step via 'magic data transfer'. It would only need to be a method by which your inside man can get access without having to compromise somebody else's password. Hell, it could be simply a misdirection to stir shit up; that's always been a viable tactic in a Cold war.
 
China is an imperialist nation.

Nice. Someone to compete with the US. At least us, citizens of the shitholes of the world, will have some leeway to negotiate a better deal when it comes to get sucked dry by a foreign power. We'll be exploited as usual, but at least the terms will be a little less worse. After the fall of the USSR, it wasn't pretty to be part of the non-white, sub-human race, you know? The US had to... pretend.... to be nice, seems that it'll have to do it again.

We the less competent by virtue of being born in the wrong place or having the wrong skin color are always cheering when the big boys start bickering. It means they'll be busy with something other than sucking us dry.
 
Last edited:
Supermicro and other tech giants are vehemently denying Bloomberg's allegations
What are they going to do admit it?
 
Up and down some. Lowest was 10.80 at 12:01..... I put in a limit order at 10.75 earlier. Almost got there.... maybe it will drop back down to there. Still got a little over 3 hours for the market today.
 
I was definitely talking about inserting at the factory and I think that'd be a different attack than selling counterfeit equipment, since I believe this equipment is, in fact, coming directly from Supermicro. that to me doesn't rise to the level of we should move our manufacturing out of china. Assuming this is true (and Bloomberg is a pretty solid organization), this does. Whether that comes back to the U.S. or it goes to Taiwan I'm good.


I can see it.
 
I watched the report and read the entire reporting.
Tbh, without actual proof I just don't buy it -- especially due to this statement: "companies do not vet the hardware they use".
That is complete and utter garbage and pretty much shows this guy doesn't know WTF he is talking about.

Typically new designs undergo a serious amount of validation before launch, but what happens after that?

In general supply agreements tend to include a clause that the supplier must tell you if they have any process changes, so you can assess what the risk of those changes are and if necessary test to make sure there are no undesired effects. There are still ongoing inspection activities, but they are usually lower grade , use small sample sizes and focus on known trouble spots

If Chinese Military Intelligence is on site, waiting until a design passes validation, and sneaking these tiny chips into the process after that fact, you may never catch it. Heck, depending on how much control they have, they could even manage the process such that they temporarily stop inserting these chips when a design change goes through, so that any validation activities performed don't catch the spy chip.

I think the chances of ongoing inspection, after validation is complete, actually catching something like this is rather small, unless like in this case, a full security audit is done.

I mean, they were saying that Amazon found even more sophisticated spy chips in their AWS datacenter in China. The chips were even tinier and thinner and were hidden in between the layers of the board! You are not going to catch that in any inspection I am aware of.

The place were it is most likely to be caught - IMHO - is in the network traffic. In order for this chip to ever do anything useful for spying, it at some point needs to communicate over the network. I'm surprised this wasn't caught sooner that way. I mean, it as using the hard coded server in the chip that they determined the extent of the affected organizations. That said, in organizations with thousands of users on a network something like this might blend into the background. Especially if they use a faked look-alike domain name trying to appear like AWS or something like that.
 
It's obvious we shouldn't have and should never again trust the Chinese. Anyone associated with doing business in any way or kowtowing to the Chinese is suspect as a traitor at this point. This stuff has got to end, we should start doing most if not all of our own fabrication etc. If we continue the status quo it's going to bite us. Mark my words.

I think this is why you see unrelated companies doing electronics manufacturing in China taking a hit on the stock market after this story went live.

Many out there are probably starting to think about what they can do to move out of China. Consumers will probably continue to not care. Unfortunately, the "caring about privacy" ship has probably sailed long ago. Slippery slope arguments rarely work, but in this case I think they were true, with people just learning to accept the step by step erosion of privacy over time. People like me complaining about it are certainly a minority.

Government and corporate users will probably hear this wakeup call though. There may be an inclination to limit the amount of electronics manufacturing done in China, which is probably why Lenovo and other unrelated Chinese computer manufacturers lost a good chunk of value over this. Short term there isn't much that can be done here though. Enough capacity to manufacture this stuff just doesn't exist elsewhere.

My guess for a best case scenario is something like this:
Short term: Western computer/electronics designers step up their inspection and test schemes for parts manufactured in China.
Medium Term: DoD and other government users start including contractual requirements that electronics are made in the U.S. and other allied states, or at the very least not China. This will drive up cost immensely. Initially this will probably result in high risk stuff being shifted to Taiwan, and more consumer stuff being shifted away from Taiwan into China, but over time it will allow the few remaining board houses in the west to increase capacity.
Long Term: Resurgent domestic (and allied) board manufacturing due to DoD contract requirements become more commonplace to the point where they become more affordable and corporate users start using them as well.

Or, a few months from now when this dies down, people choose to continue ignoring the elephant in the room, and we go back to business as usual.

Who knows.
 
https://www.businessinsider.com/countries-most-freedom-in-the-world-2018-4

BullfuckingShit. U.S.A. ranked 58. It maybe just a show but the stats are facts:

Best said at 2:50 Fast Forward

You are kidding/trolling right? This is a tv show with a specific agenda. a simple google search will give you multiple occasions where each point is proved to be either factually false, a gross exageration, or a cherry picked set of data designed to indicate something that is not fact.
 
You are kidding/trolling right? This is a tv show with a specific agenda. a simple google search will give you multiple occasions where each point is proved to be either factually false, a gross exageration, or a cherry picked set of data designed to indicate something that is not fact.


You are right, that particular clip is from a show, and thus not a very reliable source of information.

A better source would be the Democracy Index published by the Economist Intelligence Unit based on definitions of freedom as defined by organizations such as Freedom House.

58, as suggested in that TV show clip is an exaggeration, but we as a nation have fallen significantly from our once leading position during the revolution.

2017 was the first year the U.S. was not in the top segment of "Full Democracies", but instead dropped into the category of "Flawed Democracies". The U.S. is currently tied in 21st place with Italy.


Now, I know, some of you right now are already foaming at the mouth ready to yell "But the U.S. is not a democracy it is a Constitutional (or Democratic) Republic! Let me cut you off at the pass right here. This is a load of semantic bullshit.

Yes, our founding fathers devised controls in our political system to prevent the so called "tyranny by the majority", but so has just about every other democracy on the planet. Democracy doesn't mean "rule by simple majority", it just means that a people govern themselves through some form of elections, and as such our system, the Constitutional Republic is a form of Democracy, not different from democracy, just like how a Labrador is a breed of Dog, or a Ford is a brand of Car.


Only someone who has never spent time abroad could ever believe the joke that America is the "land of the free". We certainly were in the late 18th century. We have a great constitution and our nation was founded on great ideals and principles, but have failed in execution since, giving way to partisan bickering, gerrymandering, lobbying, voter disenfranchisement and many other problems that don't exist to the same extent in some better functioning democracies.

In the end, we are one of approximately 76 free democracies on earth, 20 of which are doing better than we are.

Time to shape up! Sticking our fingers in our ears and ignoring our problems isn't going to help us. The first step to fixing a problem is to acknowledging that you have it, thus all true patriots should be recognizing that we do have a problem and trying to come up with solutions, not falling over ourselves in false patriotism by yelling about how free we are, and wearing flag pins.
 
C2 comms have to occur. The data has to cross IDS / IPS to get home. I really want to see a technical write up on this before I pass judgement on SM.

lol - if your IDS/IPS is configured correctly you might catch it. There was a recent breach that was only detected because someone finally fixed the broken certificates in their IDS/IPS so it could actually inspect encrypted traffic. I suspect that's far more prevalent out there than people want to admit or suspect.
 
Who would have thought a transistor can have such a story?

Shit gets deeper.

The UK’s entire internet infrastructure is based on devices from Huawei.

Huawei was started and owned by a highup chinese intelligence officer. Yup I know the UK are fucking dumb for using them, but they did and now huawei is stealing all our data’s, especially if everyone of their devices incorporate a 1mm big chip that nobody can see.

Fuckin cunts they are. I have a great idea, why don’t we start adding huge tarrifs to chinese imports, that will fuck them up.
 
I'm still skeptical. Adding a chip to a device isn't just like sticking it on... you've got to run new traces for power, data, etc. without compromising the normal operation of the device or being detected. I need to see more information before I believe it.
 
I'm still skeptical. Adding a chip to a device isn't just like sticking it on... you've got to run new traces for power, data, etc. without compromising the normal operation of the device or being detected. I need to see more information before I believe it.
Adding a device that sits on one of the existing i2c, SPI, or UART lines wouldn't require much routing of traces. These are generally not very high speed signals and are pretty resilient to being messed with. You would probably still need to modify the PCB, I just don't think you would need much. Power and ground planes are generally everywhere.
 
You are right, that particular clip is from a show, and thus not a very reliable source of information.

A better source would be the Democracy Index published by the Economist Intelligence Unit based on definitions of freedom as defined by organizations such as Freedom House.

58, as suggested in that TV show clip is an exaggeration, but we as a nation have fallen significantly from our once leading position during the revolution.

2017 was the first year the U.S. was not in the top segment of "Full Democracies", but instead dropped into the category of "Flawed Democracies". The U.S. is currently tied in 21st place with Italy.


Now, I know, some of you right now are already foaming at the mouth ready to yell "But the U.S. is not a democracy it is a Constitutional (or Democratic) Republic! Let me cut you off at the pass right here. This is a load of semantic bullshit.

Yes, our founding fathers devised controls in our political system to prevent the so called "tyranny by the majority", but so has just about every other democracy on the planet. Democracy doesn't mean "rule by simple majority", it just means that a people govern themselves through some form of elections, and as such our system, the Constitutional Republic is a form of Democracy, not different from democracy, just like how a Labrador is a breed of Dog, or a Ford is a brand of Car.


Only someone who has never spent time abroad could ever believe the joke that America is the "land of the free". We certainly were in the late 18th century. We have a great constitution and our nation was founded on great ideals and principles, but have failed in execution since, giving way to partisan bickering, gerrymandering, lobbying, voter disenfranchisement and many other problems that don't exist to the same extent in some better functioning democracies.

In the end, we are one of approximately 76 free democracies on earth, 20 of which are doing better than we are.

Time to shape up! Sticking our fingers in our ears and ignoring our problems isn't going to help us. The first step to fixing a problem is to acknowledging that you have it, thus all true patriots should be recognizing that we do have a problem and trying to come up with solutions, not falling over ourselves in false patriotism by yelling about how free we are, and wearing flag pins.

Great points. Seems the two party state is much closer to a one party state these days - the checks and balances are out of whack.

Wonder too, if most flag pins are made in China...with or without a rice-chip mic.
 
and this is why you tightly firewall your out of band management on any server... this is why dedicated management vlans exist and why any remote user should be VPNing into such a network with plenty of security along with it...

That's great, unless you connect with a split tunnel VPN and opened up a connection for the infected server to call home from. Are the VPN tunnels being monitored to the same levels that WAN interfaces are?

I think a lot of people are doing too much in the box thinking.
 
  • Like
Reactions: DocNo
like this
That's great, unless you connect with a split tunnel VPN and opened up a connection for the infected server to call home from. Are the VPN tunnels being monitored to the same levels that WAN interfaces are?

I think a lot of people are doing too much in the box thinking.

Not going to happen if the server has no internet connection.
 
Memories.
 

Attachments

  • bw-fallujah3_iraq_oct_2002.jpg
    bw-fallujah3_iraq_oct_2002.jpg
    112.3 KB · Views: 0
  • Like
Reactions: Meeho
like this
Yup, that's why one doesn't want intel LOM's (LAN On Motherboard) or other such things. Servers, switch to broadcom or alternatives.

Even worse is the IME has access to the onboard NIC and can transmit data even when the system is powered off.
 
Back
Top