Ryzen Powerhouse PC detailed Linux review

by on 2023/01/06 at 5:28 pm
Posted In: fedora, linux

As I noted a few weeks ago, I was pondering replacing my old trusty 10 year old server with something new and quieter. I finally settled on a https://silentpc.com/powerhouse-pcs/ryzen-powerhouse build from silentpc.com. Here’s probibly more than anyone wants to hear about it.

First, I could easily have built up a new machine myself, researching and ordering parts, carefully hitting them together and cabling things, but… I have done that a number of times in my life, and now mostly find it tedious. If thats the sort of thing you enjoy, then do go for it, but personally, I’m happy to pay overhead to a place like silentpc.com to let them source all the parts, get them all working nicely and cabled well.

I went with this machine for a number of reasons:

  • I wanted something with enough drive bays for 3.5″ spinning rust so I could just move the 4 from my old machine. That would simplify migration and let me have some ‘slow’ storage for things like backups and such. This machine has 10 bays total. 1 used by a dvd drive, the others open.
  • I wanted to go with an AMD cpu. I’ve heard a lot about the current gen being fast and power efficent.
  • I wanted to start with 2 nvme drives, then add later a 4 nvme expander pci card. This MB should support that.
  • I wanted a quieter machine.

Ordering was easy. The sales folks were quick to reply to my questions about nvme expanders and video cards. I definitely did not want an nvidia card, so they arranged to add just a low end, but linux support radeon card. They estimated a 6-10 day build process before shipping, but it was less than that, and around christmas time too! They really did a banner job on packaging. The machine came in a big box with a small box of cables/etc and another box for the server itself. They carefully filled the interor of the machine with bubble wrap so nothing moved around in shipping. The case is super heavy and has noise dampening material on it. There are 3 fans, but they are all large and slowly moving (500-600rpm). Even after adding 5 3.5″ spinning drives, the fan in my computer closet is much louder than the server. The CPU heat sink is massive, but thats great, because it means it doesn’t need to run the fan at vast speeds. The power supply ( a HX850) actually powers it’s fan off when things are mostly idle!

Lets take a look at some stats from the old server:

  • Dell C1100 “cloud” node
  • 72GB memory ( 1066Mhz/DDR3 )
  • 4 3TB 3.5″ 7200 rpm SATA drives (3GB/sec) in a raid5
  • 2 Xenon L5639 @ 2.13GHz ( so 12 cores, 24 threads)

Against the new server:

  • Powerhouse Rizen PC
  • 64GB memory ( 3200Mhz/DDR4)
  • 2 Samsung SSD 990 PRO 2TB (currently in raid1)
  • 5 3TB 3.5″ 7200rpm SATA drives (6GB/sec. I had one spare, might as well add it in) in a raid5 (converting to raid6 as I write this)
  • AMD Ryzen 9 5950X 16-Core at up to 4.9GHz

So, really the only thing the old server has any more of is memory, and it’s old/slow memory anyhow. 🙂

I did need to order a few spare parts, but those were pretty easy. Got another network card (I use my main server as the firewall/gateway to my network) and some sata cables.

Like any good sysadmin, I got the new box all installed (Fedora 37 from netinstall) and configured and all ready, then set a downtime to migrate things. Everything worked fine out of the box on Linux, no problems with network cards, wireless, or anything. I did have to enable SVM in the bios to get kvm support, but that was easy enough to do. As part of this I cleaned up my computer closet, re-ran some cables and got some old UPSes batteries replaced. The downtime then was just:

  • Take down old server
  • Move network cables to new server for wireless/dsl
  • Move SATA drives from old to new server and connect
  • Add spare SATA drive and connect
  • Bring up and fix any issues

Things seemingly went fine, but then I hit something disturbing: My borg backup jobs that ran after the switch (to backup my laptop and main vm) failed. A borg check showed things were not happy. I wondered if it could be a borg bug (say that 5 times real fast), so I tried using restic. restic did manage to back up fine, but a check on it’s repo afterward showed weird corruption. Looking at raid I saw that there were a number of mismatches on the SATA raid, so I wondered if those drives just were giving up after so many years. However, I soon noticed that there were mismatches on the raid1 on the 2 brand new nvmes! My next guess was a bad memory stick, but why was it only affecting the backups/raid? A bit more digging and looking, I finally realized what it might be, and indeed a reboot brought things back to normal. The problem was that I had been playing with powertop and had done a ‘powertop –auto-tune’ and some power saving setting on some chipset device was causing all the issues. After a clean reboot and some repair/check’s, the mismatch count on both raid arrays was 0.

Overall I am quite happy with the box. It’s super fast and super quiet. I should be able to do some expanding on it over time (bump memory to 128GB, add in the nvme card, there’s even space on the back of the MB for 2 2.5: ssd sata drives if desired). There’s space for 3 more 3.5″ drives also (although I think after I add the nvme card I might drop all the spinning drives back to the old server). I would recommend/buy from again silentpc.com

Comments Off on Ryzen Powerhouse PC detailed Linux review

On generators and physics

by on 2022/12/30 at 1:20 pm
Posted In: fedora

On tuesday here we had a massive wind storm come through. 25-35mph winds with gusts up to 75mph or more, along with heavy rains and near freezing temps. It’s a pretty crazy thing to experience in the forest. 250-300ft douglas fir trees swaying 30 or 40 degrees in the winds, branches coming off and flying by. A large gust came through and broke 3 of our large trees off and pulled another one out by it’s roots. Very sad to see these big trees go.

But to bring things back to a tech focus, also on tuesday due to the storm, we lost power for about 16 hours. We had gotten a generator to handle just this situation and had run it happily for a spring outage earlier in the year. We were running on 20lb propane tanks (since they are so easy to get). In the spring we were getting about 5-6 hours of use out of a tank.

So, I fired up the generator and we got about 3 hours out of a partial tank, where I switched to our last full tank and headed into town to stock up. I exchanged the empty tank and got 3 more. That last full one was out when I got home, so I swapped another full one in and… it stopped after an hour. At first we were wondering if we had some heavy load on the generator we didn’t realize, but on checking that wasn’t the case. We looked for leaks in propane, but wait… the tank is all covered in frost and all frozen up. Here we realized the real problem.

Propane in tanks is in a liquid state, under pressure. When you open the valve, propane comes out and changes into a gas to power your whatever. This state change requires energy. Usually this is just pulled from the tank shell and everything is fine. However, when it’s cold, the tank gets colder and colder and can’t supply that energy anymore and with no gas flowing anymore, the regulator cuts off the generator.

So, at first we tried putting the tank in the garage instead of outside, wrapping it in blankets, etc. But that didn’t really help too much. So, then we got a tub and filled it with water and put the tank in there. That helped quite a lot, we managed to get about 4 hours out of a tank then. At the end the tank had a 1 inch or so ‘wrapper’ of ice, but it did get the tank lasting longer.

So, whats the solution here? Well, I think getting just one larger tank (say a 100lb one) would help a lot as it will have a ton more surface area. Not as easy you move around, but oh well. Also, they make apparently powered heating blankets for the smaller tanks. I’ll probibly pick up one of those in the short term. So, lesson learned: temperature has a lot to do with how well your propane generator will work.

└ Tags: , , ,
Comments Off on On generators and physics

Some thoughts on a new home server

by on 2022/12/13 at 1:10 pm
Posted In: fedora, linux

I’ve been spending some of my time off in the last few days pondering replacing my old reliable home server with something new and shiny. I figured this might be a good time to write up some thoughts around this.

So, the first question that I am sure leaps to mind for people is: Home server? why on earth do you want one of those! Move it to “The Cloud”! Of course doing so would indeed have a number of advantages:

  • Better bandwith
  • No need to hassle with hardware, someone else would do that
  • Less noise and power usage at home
  • Depending on how deep in the clouds you go: less hassle running services

On the other hand it has real disadvantages to me:

  • No “real life” home setup to test/try/figure things out.
  • Never really 100% sure who has/owns/can do things with your data
  • Ability to mess with hardware, which can be kind of fun.
  • I have a small list of close friends who I provide services to. It’s fun to keep in touch with them that way and have something I can do for them.
  • Ability to mess with running a bunch of services, which can be kind of fun.
  • Paying a cloud provider recurring fees for something I could just buy once and not pay for over and over again seems like it could be a win, depending on the fees.

Someday I might give up and move things, but it’s not come fully to that yet. Email has been slowly getting more difficult to run on a non gigantic domain, but I’ve managed to overcome so far, so I will keep going until that becomes completely untenable. I really like having my data close by and knowing that I can go fix some problem when it happens. It’s also been a while, but I want to look at spinning up a home OpenShift instance so I can dig into it more and learn more about the low level parts of it. Might need to use OKD or k3s or something instead of OpenShift, but should let me find out more about how ks8 works.

All that background said, lets look at my current home server. It’s a Dell PowerEdge C1100/CS24-TY. I got it from https://deepdiscountservers.com long long long ago, along with another identical server. You can really get pretty great stuff there. It’s basically all the old compute that cloud companies have aged out. So, they are usually older, but tons of memory and disk and cpu. These ones I got have 72GB memory, 24 cpu threads, and 4 3.5″ hot swap drive bays in the front. The second one I got I used for a long time as a test machine, but it has a slightly too old cpu to do power management, so it’s really really loud. The main server does do some power management, but it’s pretty loud too. In my current house I have a closet for computer stuff, but even with the door closed I am near enough to it that I can hear the server running. Of course I can also usually hear the fridge in the kitchen running too. The drives I currently have are 3TB 7200 rpm hitachi’s. Which have also been quite reliable. The server has a pci card in it for some more network ports. It serves as my main firewall / virthost / storage server.

So, why replace it? Well, it was made in the fall of 2012. Yes, thats 10 years old now. Thats ages in computer hardware. It’s slow. The cpu is pretty slow and the storage is super slow. It’s running the 7200rpm spinning disks on a 3GB/sec sata bus (They can do 6GB/s). Taking backups or moving a bunch of things or running a postgresql vacuum just takes ages. It’s also loud. Not earthshatteringly so, and like I mentioned our fridge is also kinda loud, but there’s a lot of times when the fridge compressor is off and I can hear the server distinctly. Finally, it’s fun to look at things and then install and assembe them. Computer geeks gotta geek. Also, this is perhaps a chance for me to play with some things I haven’t yet, like perhaps moving over to a AMD cpu instead of intel or raid on nvme, etc

So, my first thought was to just get another rackmount from deepdiscountservers, which would work fine, but it would be intel based, basically just a newer version of what I have now with more memory and cpus. The cpus would be intel and while newer servers are likely to do throttling better, I don’t think the noise would be all that much lower. rack mount servers are just not designed to be quiet.

Next, I poked around on the net and ran accross silentpc.com, which has some interesting computers on offer. I focused in on the “Powerhouse Ryzen PC” box. It’s a tower case, which is not ideal, but I’m sure I can fit it in somewhere. It’s a Ryzen cpu, a power supply that can power completely off if things are idle, super quiet fans, etc. It’s got enough room so I can move my existing 4 drives over to it (and add in a 5th that I have I was keeping for a spare). Only 2 NVME slots available, but… that takes me into an aside I had:

Most motherboards these days I have seen have just a few NVME slots on them. However, they make PCIe cards that have NVME slots (one, two, or four). The four slot NVME’s are interesting. You need to have a motherboard that supports “pci bifurication” on the slot you are putting it in. If you don’t, you can only see one drive and thats it. If you do, the motherboard takes the x16 slot and carves it into 4 x4 slots and you see all the drives.

From what I have been able to gather the Powerhouse Ryzen PC has a motherboard that has 1 pcie slot that can do bifurcation (but I asked them in email to make sure). If so, then I can get it with 2 NVME’s and raid1 them for now, move the 3.5″ drives over with most of my data, and then down the road I can get a PCIe 4 NVME card and stick 4 NVME’s in there and raid 6 them with the 2 on the MB and then perhaps retire the spinning drives. 🙂 Sadly, their web interface seems to only offer nvidia cards (which I really don’t want), but I asked them in email and they can indeed do other cards. So, waiting to hear back, but I think this might work out nicely for a new box. If it does, I’m also thinking about moving the existing 1U boxes out to the garage and see if I can set them up with a wake on lan or the like so I can use them if I need to test something.

Looking forward to tinkering with it (or looking more if this one doesn’t pan out).

└ Tags: , , ,
Comments Off on Some thoughts on a new home server

New phone, who dis?

by on 2022/11/04 at 2:13 pm
Posted In: fedora, linux

In october, I went on 2 trips and after that several things became very clear: First, covid is still out there and you can still get it (as I did) and that my current trusty phone that I had been using for the last 7 years finally needed to be replaced.

The old phone is a one plus 3t and it’s been a great phone. I currently have been running /e/ on it (with no google) and it’s been fine. Unfortunately, the years have now taken their toll on the battery. When traveling I had to put it on super battery save just to have any battery left after a day away from a charger. On super battery save it’s slow slow slow, bordering on unusable. The sad thing is, if I could replace the battery I could probibly be fine with this phone for more years. It’s true it doesn’t take super great pictures, only has 64GB space, and has a bunch of scratches now. But alas, the battery is non replaceable, so I decided I needed to do something.

Of course the first thing I looked at was just using one of the 2 pinephones or 1 pinephone pro I already have. Sadly, they just aren’t good enough for a daily driver for me. They are slow, the battery life is also bad (on the pro at least), and… the biggest problem: The camera is just not good at all. I end up taking a lot of pictures and I really need them to be viewable. Finally, despite lots of work, there’s still a bunch of things non upstreamed, so I would have to run a custom kernel and a bunch of other non upstreamed parts.

Next, there are now some phones you can buy with /e/ pre-installed. There’s the Murena one and Teracube 2e. The one has more storage space, but otherwise they have less ram than my oneplus 3t. 🙂 This was a tempting option, but /e/ hadn’t been impressing me of late. Updates were few and far between, I had to do a bunch of tinkering to get on the latest stream (based on android 11).

The fairphone3/4 are interesting, but don’t seem to be available in the US at all. Of course I am sure I could get one, but likely it wouldn’t work with US telco’s.

I wanted to get something that would have a nice long life of updates also.

Finally I ended up deciding on just getting a unlocked pixel 7 and going with grapheneos on it. I deliberately choose the 7 over the 7 pro because it’s slightly smaller (but still big) and from all reviews I read had a better battery life. The pixel 7 gets 5 years of updates, which is about as good as you can do these days.

Why grapheneos? Well, I did not want to go back to being tied to google if I could help it and /e/ doesn’t really support any very modern phones. The free phone os’es (postmarketos, mobian, etc) also don’t support pretty new hardware either. However, I figure after a few years there’s a good chance something like a pixel will be supported by more of those and I can choose to jump to one of them if I want. grapheneos is basically ASOP (the “upstream” android) with a bunch of security enhancements added to it. The install process was pretty painless, but I did hit one problem where I tried to install with the web installer using firefox, then switched to chrome and got an error. I finally figured out that I forgot to close the firefox tab out and it was keeping the webusb locked so the other browser couldn’t install. 😉

Install went fine after that and I installed f-droid and got my applications and data moved over to the new phone. The biggest headache was of course signal. I had been using it for sms/mms after my last /e/ re-install, but they are dropping support now, so I had to export my sms’s back out of it to get them moved over. The export function doesn’t tell you that it doesn’t handle duplicates and you should wipe your sms db first, so I ended up with 2x my sms messages. Finally got that transfered over and signal deleted. signal could have been a great app, but they seem determined to made decisions that will drive them into irrelevance now. It’s sad.

Anyhow, I hope the pixel 7 will last me a few more years until I can get a modern phone and put Fedora on it. 🙂

Comments Off on New phone, who dis?

Onlykey DUO

by on 2022/05/08 at 1:47 pm
Posted In: fedora, linux

Last year, I backed the onlykey DUO on kickstarter: https://www.kickstarter.com/projects/timsteiner/onlykey-duo-portable-protection-for-all-of-your-devices It seemed like a interesting device and I like that it’s fully opensource, unlike modern yubikeys.

The device finally arrived last month, and I’ve had a chance to play around with it some. Sadly, I don’t think it’s going to replace my yubikey anytime soon.

On the good side: The device itself is nicely constructed. It has a multicolored led on it that indicates which profile is in use (There are 4: green, blue, yellow, purple). It’s got 2 buttons on the end, so you can press one or the other or both at the same time and long or short presses for different slots. That means each profile has 6 ‘slots’ for a total of 24 in all 4 profiles. You can set a pin to lock the key which you have to enter before using it, along with a ‘self destruct’ pin that will wipe all configuration when entered.

On the bad side however, there’s a fair bit. The software to manage the onlykey is provided as either a ubuntu .deb or a snap. I tried to get the snap working with no luck at all, and ended up unpacking the deb to get things working. I looked into making a Fedora package but it’s a node app and has a pile of deps.

Next, I tried to enroll a otp for our Fedora account system, but found that the TOTP secret wouldn’t work. Further investigation showed that the onlykey NEO only supports sha1 for TOTP secrets and our account system uses SHA512. ;( There’s a old closed ticket about this on the onlykey firmware repo: https://github.com/trustcrypto/OnlyKey-Firmware/issues/101

There’s also no way to generate a ssh private key on the device (like you can using the opensc support on a yubikey). You can generate ecdsa sk openssh keys, which is great, but not too useful to me yet as RHEL7 and RHEL8 don’t support them.

So, at this point I would not recommend these devices unless you don’t need to interact with the Fedora account system or want to use the device with a Fedora linux install.

Comments Off on Onlykey DUO