FreeBSD disk io

[joske]

For FreeBSD disk io, these are the disks in my system:

[jos@bsd ~]$ ls /dev/ada*
/dev/ada0	/dev/ada0p2	/dev/ada0p4	/dev/ada1s1	/dev/ada1s3	/dev/ada2p1
/dev/ada0p1	/dev/ada0p3	/dev/ada1	/dev/ada1s2	/dev/ada2	/dev/ada2p2
[jos@bsd ~]$ ls /dev/nvd0*
/dev/nvd0	/dev/nvd0p1	/dev/nvd0p2	/dev/nvd0p3	/dev/nvd0p4

nvd0 is an nvme disk with Linux
ada0 is a sata disk with FreeBSD (GhostBSD)
ada1 is a sata disk with win10
ada2 is a sata disk with macOS

This is the debug output of the collect_disk method:

2021/11/16 (22:03:46) | DEBUG: dev nvd0 read=7201280 write=0
2021/11/16 (22:03:46) | DEBUG: dev ada1 read=1096481280 write=1747473408
2021/11/16 (22:03:46) | DEBUG: dev ada2 read=37594112 write=0
2021/11/16 (22:03:46) | DEBUG: dev ada3 read=5676544 write=0
2021/11/16 (22:03:46) | DEBUG: dev cd9 read=0 write=0
2021/11/16 (22:03:46) | DEBUG: dev pass4 read=0 write=0
2021/11/16 (22:03:46) | DEBUG: dev pass5 read=0 write=0
2021/11/16 (22:03:46) | DEBUG: dev pass6 read=0 write=0
2021/11/16 (22:03:46) | DEBUG: dev pass7 read=0 write=0
2021/11/16 (22:03:46) | DEBUG: dev pass8 read=0 write=0

Seems to match a little bit, although for the ada ones, it seems to be off by 1. Doesn't seem to break down the stats per partition/slice, but per total disk.

The cd9 entry could be the dvd drive. No idea what the pass devices are.

[joske]

Ah the number does not belong to the disk, but to the index in the array. I have 5 pass devices:

[jos@bsd ~]$ ls -a /dev/pass*
/dev/pass0	/dev/pass1	/dev/pass2	/dev/pass3	/dev/pass4

According to pass(4) these are device nodes for Common Access Method Storage subsystem:

[jos@bsd ~/Projects/btop]$ camcontrol devlist 
<Samsung SSD 850 EVO 500GB EMT01B6Q>  at scbus0 target 0 lun 0 (pass0,ada0)
<ASUS DRW-24D5MT 2.00>             at scbus1 target 0 lun 0 (cd0,pass1)
<KINGSTON SA400S37120G 03090005>   at scbus2 target 0 lun 0 (pass2,ada1)
<Samsung SSD 850 EVO 250GB EMT02B6Q>  at scbus3 target 0 lun 0 (pass3,ada2)
<AHCI SGPIO Enclosure 2.00 0001>   at scbus6 target 0 lun 0 (ses0,pass4)

The nvme drive is missing from this list however.

[joske]

Ok, so the trick is to use unit_number, this gives the correct device:

2021/11/16 (22:24:12) | DEBUG: dev nvd0 read=7202304 write=0
2021/11/16 (22:24:12) | DEBUG: dev ada0 read=1099911680 write=2275030016
2021/11/16 (22:24:12) | DEBUG: dev ada1 read=37595136 write=0
2021/11/16 (22:24:12) | DEBUG: dev ada2 read=5677568 write=0
2021/11/16 (22:24:12) | DEBUG: dev cd0 read=0 write=0
2021/11/16 (22:24:12) | DEBUG: dev pass0 read=36 write=0
2021/11/16 (22:24:12) | DEBUG: dev pass1 read=0 write=0
2021/11/16 (22:24:12) | DEBUG: dev pass2 read=0 write=0
2021/11/16 (22:24:12) | DEBUG: dev pass3 read=0 write=0
2021/11/16 (22:24:12) | DEBUG: dev pass4 read=0 write=0

Now this matches the devices. However, as the BSD system uses zfs, this does not map to anything that you get from
getmntinfo().

Also strange that the Linux disk has a high read count, as I've not even mounted it in BSD.

[aristocratos]

@joske

Now this matches the devices. However, as the BSD system uses zfs, this does not map to anything that you get from
getmntinfo().

Yeah, that would be the crux, may need a third function (or file) that can correlate some piece of data between the two sets.

[joske]

I've updated ghostbsd, and system became very unstable. Now it doesn't even boot anymore. I can access the files from linux using the linux zfs drivers, but it just doesn't boot anymore, not in single user, not safe mode, nothing. Will have to re-install, so will be out of business until I have time to do that...

If it proposes updates, don't install them!

[joske]

update: apparently some BIOS setting broke it. I had the same results with booting from USB drive, so I reset all BIOS settings, and now back ok.

I've posted a question regarding disk io on the FreeBSD forums in the development section. Hopefully someone can help.

[aristocratos]

I reset all BIOS settings, and now back ok.

That's a relief.

I've posted a question regarding disk io on the FreeBSD forums in the development section. Hopefully someone can help.

Ah, smart move. Can you post a link to the thread so I can follow along.

[joske]

https://forums.freebsd.org/threads/disk-io.83011/

…

On Sat, 20 Nov 2021, 14:05 aristocratos, ***@***.***> wrote: I reset all BIOS settings, and now back ok. That's a relief. I've posted a question regarding disk io on the FreeBSD forums in the development section. Hopefully someone can help. Ah, smart move. Can you post a link to the thread so I can follow along. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#156 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEB7R7OI3N4WQIKCUJQGQDUM6MIFANCNFSM5IFIOTXQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[joske]

@aristocratos I've implemented 'regular' filesystems using devstat API (a bit special as this gives per block device not per partition/mountpoint), so several mountpoints can match. I've implemented the ZFS filesystems with sysctl output parsing.

The debug logs show it does calculate for all, but the io graph only shows root. Can you have a look?

[joske]

and this is valgrind output for this code:

[jos@bsd ~/Projects/btop]$ valgrind --leak-check=full --track-origins=yes bin/btop --debug
==6516== Memcheck, a memory error detector
==6516== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==6516== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==6516== Command: bin/btop --debug
==6516== 
==6516== Syscall param ioctl(generic) points to uninitialised byte(s)
==6516==    at 0x50475AA: ioctl (in /lib/libc.so.7)
==6516==    by 0x4FA55C6: fdevname_r (in /lib/libc.so.7)
==6516==    by 0x4FC433F: ttyname (in /lib/libc.so.7)
==6516==    by 0x4C2E6D: Term::init() (btop_tools.cpp:116)
==6516==    by 0x4EC982: main (btop.cpp:741)
==6516==  Address 0x7fc000464 is on thread 1's stack
==6516==  in frame #1, created by fdevname_r (???:)
==6516==  Uninitialised value was created by a stack allocation
==6516==    at 0x4FA5590: fdevname_r (in /lib/libc.so.7)
==6516== 
==6516== Syscall param ioctl(generic) points to uninitialised byte(s)
==6516==    at 0x50475AA: ioctl (in /lib/libc.so.7)
==6516==    by 0x4FA55C6: fdevname_r (in /lib/libc.so.7)
==6516==    by 0x4FC433F: ttyname (in /lib/libc.so.7)
==6516==    by 0x4C2E98: Term::init() (btop_tools.cpp:116)
==6516==    by 0x4EC982: main (btop.cpp:741)
==6516==  Address 0x7fc000464 is on thread 1's stack
==6516==  in frame #1, created by fdevname_r (???:)
==6516==  Uninitialised value was created by a stack allocation
==6516==    at 0x50E1590: ??? (in /lib/libc.so.7)
==6516== 
==6516== 
==6516== HEAP SUMMARY:
==6516==     in use at exit: 779,855 bytes in 62 blocks
==6516==   total heap usage: 104,124 allocs, 104,062 frees, 62,350,234 bytes allocated
==6516== 
==6516== 3,464 bytes in 1 blocks are definitely lost in loss record 46 of 52
==6516==    at 0x484B8F4: malloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==6516==    by 0x4C90439: devstat_getdevs (in /lib/libdevstat.so.7)
==6516==    by 0x4FC30F: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:511)
==6516==    by 0x4FF31B: Mem::collect(bool) (btop_collect.cpp:726)
==6516==    by 0x4F8F00: Shared::init() (btop_collect.cpp:171)
==6516==    by 0x4ECCC8: main (btop.cpp:773)
==6516== 
==6516== LEAK SUMMARY:
==6516==    definitely lost: 3,464 bytes in 1 blocks
==6516==    indirectly lost: 0 bytes in 0 blocks
==6516==      possibly lost: 0 bytes in 0 blocks
==6516==    still reachable: 776,391 bytes in 61 blocks
==6516==         suppressed: 0 bytes in 0 blocks
==6516== Reachable blocks (those to which a pointer was found) are not shown.
==6516== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6516== 
==6516== For lists of detected and suppressed errors, rerun with: -s
==6516== ERROR SUMMARY: 147 errors from 17 contexts (suppressed: 0 from 0)
[jos@bsd ~/Projects/btop]$ 

[aristocratos]

@joske

The debug logs show it does calculate for all, but the io graph only shows root. Can you have a look?

I believe this behavior would be fine considering this how bpytop operates for both freebsd and osx.

Have only had time to take a quick glance at the code, but I can't say I'm a big fan of the os call to sysctl which both have security and performance implications. Should also RAII wrap the popen since there is code that could throw exceptions inside that scope.

Edit: Saw that you got some responses in the freebsd forums thread, will take a look when I have time if we can get equivalent information programmatically some way from the source to the programs they suggested.

[joske]

@aristocratos I don't think there's a way to do this programmatically :-(. Nobody even on FreeBSD dev forum seems to know.

[joske]

@aristocratos implemented RAII and iterate over all ZFS pools