Linux Kernel Tuning: page allocation failure

If you start seeing these errors it means your server or instance started running out of kernel memory. [10223.291166] java: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null) [10223.301794] java cpuset=/ mems_allowed=0-1 [10223.307211] CPU: 29 PID: 19395 Comm: java Not tainted 4.14.154-99.181.amzn1.x86_64 #1 [10223.315658] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006 [10223.322004] Call Trace: [10223.325230] <IRQ> …

AWS EBS Storage Performance Notes – Instance throughput vs Volume throughput

I just wanted to write a couple lines/guidance on this regard as this is a recurring question when configuring storage, not only in the cloud, but can also happen on bare metal servers. What is throughput on a volume? Throughput is the measure of the amount of data transferred from/to a storage device per time …

Linux Kernel Tuning: failed to alloc buffer for rx queue

If we put enough pressure over the ENA network driver, we’ll start seeing these “failed to alloc buffer for rx queue” messages on the ‘dmesg‘ output. This message will raise when the napi handler fails to refill new Rx descriptors, typically due to lack of memory. This situation might lead to performance decrease, given that …

Linux Kernel Tuning: task blocked for more than 120 seconds

This might be old school, and maybe even boring reading. But, if you concern about performance on Linux servers, at some point, you will have to have a look to the kernel messages. The problem: When we run very stressful jobs running on large servers (large number of CPU’s and RAM memory), where IO activity …