A conundrum – ESXi and solaris zfs

Despite the change to a full hypervisor I wanted to keep using zfs to manage the storage of datafiles. Partly because it seems like a great system and the reporting and management is excellent and simple and partly because that’s what I started out with and I’m too bloody minded to change.

One of my inspirations for this came from solarisinternals.com who have moved to a similar setup.  There they have setup zfs to access the disks on esx server using ‘raw disk mode’ which is obviously exactly what I want isn’t it? This would be my preferred route. I know it goes against the virtual ethos but I could whack the drives into any solaris box and ‘zfs import’ them in the event of a hardware failure. I gives me options to keep the drives without having to offload all the data and re-format; if I run into problems with ESXi in the future.

Well NOT SO FAST…..it’s a case of needed to do the research and read the small print. ESXi 4.0 doesn’t support virtual machines accessing raw disk devices (‘raw disk mode’). It seems that older versions (or maybe other vmware products do or did). RDM or ‘Raw disk Mapping’ is a supoprted option in ESXi 4 but that refers to mapping onto raw disks over a SAN (NOT LOCALLY).

I have created an opensolaris 2009.06 virtual machine running on the hypervisor. The root pool or system disk of this is infact a .vmdk file sitting on the internal mirrored pair of drives sitting in the server. My intention was to add additional drives that would be managed directly under opensolaris. BUT this just doesn’t seem possible…ESXi 4.0 doesn’t allow raw device access or direct disk access.

Research is ongoing I have two choices it seems.

1. Use the hardware raid capabilities of the SAS/SATA RAID cards – Then just use ZFS to manage quotas/snapshots and management stuff. BUT I’m nervous about recovering these should a controller fail (I’m left with a situation where in order to recover the data I’d need to buy a very specific and very expensive RAID controller – or wait ages until the right thing came up on ebay). Also RAID-Z in ZFS removes write-hole errors.
2. Create virtual disks on the actual disks and use zfs to manage these as if they were actual disks. I guess I can see a disaster recover route for this option. The disk could be hooked out and connected to any sata controller and then read from within ESXi (I think) but I need to check that. This would have to be slower wouldn’t it?
3. Forget zfs completely. Use hardware RAID and create another virtual machine which is a small footprint freenas or similar box let that take care of all the file serving work. Still need to think about a possible route for recovering the data in the event of a hardware failure.
4. Find work around – there is always a work around!
5. Sod it – switch to hyper-v which does seem to support it!

ESXi access local disks as a raw device – workarounds
1. Use vmkfstools: There do seem to be worked examples
here………http://www.hardforum.com/showthread.php?t=1441318
and here…http://www.hardforum.com/showthread.php?t=1429640
2. Edit configuration files by hand
discussed here… http://communities.vmware.com/thread/145589?tstart=0&start=0

Decisions, decisions…

140 Comments

  1. Shawn Said,

    December 10, 2014 @ 7:48 pm

    .

    good info!!…

  2. Marcus Said,

    December 10, 2014 @ 10:08 pm

    .

    ñïñ çà èíôó….

  3. cory Said,

    December 11, 2014 @ 1:56 am

    .

    áëàãîäàðåí….

  4. Milton Said,

    December 11, 2014 @ 6:57 pm

    .

    tnx!…

  5. Gabriel Said,

    December 11, 2014 @ 10:36 pm

    .

    áëàãîäàðþ….

  6. Jorge Said,

    December 12, 2014 @ 11:26 pm

    .

    ñïñ!!…

  7. ricardo Said,

    December 13, 2014 @ 2:51 am

    .

    ñýíêñ çà èíôó….

  8. edward Said,

    December 13, 2014 @ 2:44 pm

    .

    thanks for information….

  9. Shannon Said,

    December 17, 2014 @ 7:12 pm

    .

    ñïàñèáî çà èíôó!…

  10. tyler Said,

    December 19, 2014 @ 7:33 am

    .

    good!!…

  11. Lee Said,

    December 20, 2014 @ 8:26 am

    .

    good info….

  12. Benjamin Said,

    December 23, 2014 @ 6:08 am

    .

    áëàãîäàðñòâóþ!!…

  13. Christopher Said,

    December 23, 2014 @ 11:20 am

    .

    ñïñ!…

  14. Fredrick Said,

    December 24, 2014 @ 9:44 am

    .

    ñïàñèáî!…

  15. francisco Said,

    January 15, 2015 @ 1:24 am

    .

    ñýíêñ çà èíôó!…

  16. Claude Said,

    January 16, 2015 @ 3:33 pm

    .

    tnx….

  17. Dustin Said,

    January 17, 2015 @ 1:01 am

    .

    áëàãîäàðþ!!…

  18. James Said,

    January 17, 2015 @ 2:36 am

    .

    ñïàñèáî çà èíôó!!…

  19. Jay Said,

    January 17, 2015 @ 10:21 am

    .

    áëàãîäàðþ….

  20. rex Said,

    January 17, 2015 @ 10:51 am

    .

    thanks!…

  21. Walter Said,

    January 17, 2015 @ 11:22 am

    .

    ñïàñèáî çà èíôó!!…

  22. jaime Said,

    January 23, 2015 @ 9:43 pm

    .

    hello….

  23. matt Said,

    January 23, 2015 @ 11:23 pm

    .

    thank you!…

  24. Gerard Said,

    January 25, 2015 @ 6:23 am

    .

    ñïñ….

  25. randall Said,

    January 25, 2015 @ 11:21 pm

    .

    thanks!…

  26. dave Said,

    January 25, 2015 @ 11:55 pm

    .

    áëàãîäàðñòâóþ!…

  27. everett Said,

    January 29, 2015 @ 5:36 am

    .

    thank you!…

  28. clayton Said,

    January 30, 2015 @ 12:42 pm

    .

    thank you!!…

  29. Aaron Said,

    January 31, 2015 @ 12:55 am

    .

    ñýíêñ çà èíôó!!…

  30. karl Said,

    January 31, 2015 @ 4:41 pm

    .

    hello….

  31. Julian Said,

    February 1, 2015 @ 2:14 pm

    .

    good info!!…

  32. William Said,

    February 1, 2015 @ 2:32 pm

    .

    thanks!…

  33. Jon Said,

    February 1, 2015 @ 3:06 pm

    .

    thanks….

  34. harry Said,

    February 2, 2015 @ 6:00 am

    .

    ñïñ!!…

  35. Alfredo Said,

    February 4, 2015 @ 12:37 am

    .

    ñïàñèáî çà èíôó!!…

  36. jim Said,

    February 7, 2015 @ 8:05 pm

    .

    ñïñ….

  37. darren Said,

    February 8, 2015 @ 10:06 pm

    .

    thank you!!…

  38. Gerald Said,

    February 10, 2015 @ 12:06 am

    .

    thanks for information!!…

  39. troy Said,

    February 13, 2015 @ 8:03 am

    .

    ñïàñèáî!!…

  40. Orlando Said,

    February 13, 2015 @ 11:45 am

    .

    ñýíêñ çà èíôó!!…