USB Devices as VMFS Datastore in vSphere ESXi 6.5 https://www.virten.net/2016/11/usb-devices-as-vmfs-datastore-in-vsphere-esxi-6-5/

 In ESXi 6.5, there are some changes concerning devices connected with USB. The legacy drivers, including xhci, ehci-hcd, usb-uhci, and usb-storage have been replaced with a single USB driver named vmkusb. The new driver has some implications if you are trying to use USB devices like USB sticks or external hard disks as VMFS formatted datastore.

Some people have reported that they have issues with USB Datastores since ESXi 6.5. I've tried to reproduce and fix those problems. This post explains the changes in the new version and how to create VMFS 5 or VMFS6 formatted USB devices as datastore on your ESXi host.

Please remember that using USB Datastores is not supported by VMware so if you have problems, do not open service requests for that.

Device Naming Changes in ESXi 6.5

Which I noticed first was that the devices naming in 6.5 have been changed. In ESXi 6.0, all USB devices are recognized as mpx.vmhbaXX while in 6.5 some are listed as naa.XXX. Here is an example of a Samsung M.2 flash drive connected with a USB3 adapter:

# ESXi 6.0u2
-rw-------    1 root     root     128035676160 Nov 27 15:57 naa.5000000000000001

# ESXi 6.5
-rw-------    1 root     root     128035676160 Nov 27 18:16 mpx.vmhba36:C0:T0:L0

However, not all devices are affected. My 16GB flash drives shows up with the old mpx.vmhba naming on both versions:

# ESXi 6.0u2
-rw-------    1 root     root     16022241280 Nov 27 16:02 mpx.vmhba34:C0:T0:L0

# ESXi 6.5
-rw-------    1 root     root     16022241280 Nov 27 16:18 mpx.vmhba34:C0:T0:L0

The new naming does not have an impact on USB datastores, despite it might happen that you overlook the USB device when checking the /dev/disks/ directory. You can still partition and format USB devices with VMFS like in previous versions, just use the new name.

Existing Datastores are not mounted automatically

A common problem which I've heard multiple times, but only seen since ESXi 6.5, is that when you connect a flash drive that already contains a VMFS datastore, the datastore does not show up. You can see the device in /dev/disks but the datastore remains offline. The vmkernel.log hints the problem:

2016-11-27T10:25:05.885Z cpu0:65593)VMWARE SCSI Id: Id for vmhba34:C0:T0:L0
0x50 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x41 0x53 0x32 0x31 0x31 0x35
2016-11-27T10:25:05.885Z cpu0:65593)ScsiDeviceIO: 9120: Get VPD 80 Inquiry for device "naa.5000000000000001" from Plugin "NMP" failed. Not supported
2016-11-27T10:25:05.885Z cpu0:65593)ScsiDeviceIO: 9167: Get VPD 86 Inquiry for device "naa.5000000000000001" from Plugin "NMP" failed. Not supported
2016-11-27T10:25:05.887Z cpu0:65593)ScsiDeviceIO: 7851: Could not detect setting of QErr for device naa.5000000000000001. Error Not supported.
2016-11-27T10:25:05.891Z cpu0:65593)ScsiDeviceIO: 8365: Could not detect setting of sitpua for device naa.5000000000000001. Error Not supported.
2016-11-27T10:25:05.898Z cpu1:65562)ScsiDeviceIO: 2948: Cmd(0x439500b60000) 0x1a, CmdSN 0x238 from world 0 to dev "naa.5000000000000001" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11136: Device naa.5000000000000001:1 detected to be a snapshot:
2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11143: queried disk ID: <type 2, len 14, lun 0, devType 0, scsi 0, h(id) 4847995147196820166>
2016-11-27T10:25:05.905Z cpu0:65593)LVM: 11150: on-disk disk ID: <type 0, len 11, lun 0, devType 0, scsi 0, h(id) 8386732072863042940>
2016-11-27T10:25:05.914Z cpu0:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 40, Parameter: 0x43029e372700, Registered!
2016-11-27T10:25:05.914Z cpu0:65593)ScsiEvents: 300: EventSubsystem: Device Events, Event Mask: 200, Parameter: 0x43029e372700, Registered!
2016-11-27T10:25:05.914Z cpu0:65593)ScsiDevice: 4265: Successfully registered device "naa.5000000000000001" from plugin "NMP" of type 0

The datastore obviously gets detected as snapshot. I don't know why and I've only seen this in ESXi 6.5 but it's very easy to fix. Get a list of VMFS snapshots with ESXCLI:

# esxcli storage vmfs snapshot list
583b1a72-ade01532-55f6-f44d30649051
 Volume Name: usbflash
 VMFS UUID: 583b1a72-ade01532-55f6-f44d30649051
 Can mount: true
 Reason for un-mountability:
 Can resignature: true
 Reason for non-resignaturability:
 Unresolved Extent Count: 1

Use the VMFS UUID to mount the datastore:

# esxcli storage vmfs snapshot mount -u 583b1a72-ade01532-55f6-f44d30649051

The datastore should now show up within /vmfs/volumes and the Embedded Host Client.

Create VMFS5/6 Datastore on USB drives

Creating VMFS5 or VMFS6 datastores works just as in previous versions:

1. Connect to the ESXi host with SSH
2. Stop the USB arbitrator service. This service is used to passthrough USB device from an ESX/ESXi host to a virtual machine. (When disabling it, you can no longer passthrough USB devices to VMs)

   ~ # /etc/init.d/usbarbitrator stop

3. (optional) Use this command to permanently disable the USB arbitrator service after reboot.

   ~ # chkconfig usbarbitrator off

4. Plug in the USB device to your ESXi host. While connecting the USB device you can either watch /var/log/vmkernel.log to identify the device name or identify it within /dev/disks.
   Devices are either named mpx.vmhbaXX or with an naa.X number.

   ~ # ls /dev/disks/

5. Write a GPT label to the device (Assuming that the Device ID is naa.5000000000000001)

   ~ # partedUtil mklabel /dev/disks/naa.5000000000000001 gpt

6. To create a partition you need to know the start sector, end sector, which depends on the device size and the GUID.
   The start sector is always 2048
   The GUID for VMFS is AA31E02A400F11DB9590000C2911D1B8
   The end sector can be calculated with the following formula (Use the numbers from getptbl):

   ~ # partedUtil getptbl /dev/disks/naa.5000000000000001
   gpt
   15566 255 63 250069680

   15566 * 255 * 63 - 1 = 250067789

   You can also calculate the end sector with the following command:

   ~ # eval expr $(partedUtil getptbl /dev/disks/naa.5000000000000001 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1
   250067789

7. Create the VMFS partition (Replace with your end sector)

   ~ # partedUtil setptbl /dev/disks/naa.5000000000000001 gpt "1 2048 250067789 AA31E02A400F11DB9590000C2911D1B8 0"

8. Format the partition with VMFS5 or VMFS6

   # VMFS v5
   ~ # vmkfstools -C vmfs5 -S USB-Datastore /dev/disks/naa.5000000000000001:1
   
   # VMFS v6
   ~ # vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/naa.5000000000000001:1

The USB-Datastore should now appear in your datastores view.

This is how your command output should look like:

[root@esx4:~] partedUtil mklabel /dev/disks/naa.5000000000000001 gpt
[root@esx4:~] eval expr $(partedUtil getptbl /dev/disks/naa.5000000000000001 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1
250067789
[root@esx4:~] partedUtil setptbl /dev/disks/naa.5000000000000001 gpt "1 2048 250067789 AA31E02A400F11DB9590000C2911D1B8 0"

 92274688=44G

92274688/2048=445056

44*1024*1024*1024 / 512

99i

Reduce size

 Yes - you can cut a vmdk to a new desired size in place - without the need of a copy / migration.

Assuming that you already adjusted the descriptorsize to the new size we will use that value for the next calculations.
So lets take this example .... only showing the relevant line of the descriptorfile.
RW 4508876800 VMFS "name-flat.vmdk"
Enter this value into a calculator and check if the size in sectors is a multiple of 1MB.
4508876800 sectors / 2048 = 2201600 MB.
Using flat.vmdk - sizes that are not a multiple of 1MB is allowed but not ideal at all.
So rather adjust the new descriptor and pick a value that is a multiple of 1MB.
In this case the value of 2201600 is fine.
To use dd in "scissor-mode" you must inject at least one block of data at a given skip-value and MUST not use the "conv=notrunc" option.
The dd command needs the following parameters:
bs= blocksize (we will use 1Mb here which corresponds to bs=1M)
if= inputfile (we can use either /dev/zero or a specially created file I call "cut.bin" of exactly 1MB)
of= outputfile (here we assign the flat.vmdk we want to cut)

seek= this is the offset in the outputfile where we want to cut MINUS 1MB)
I used to inject empty blocks created from /dev/zero in the past but learned a safer approach during last years ....

So I nowadays use a "cut.bin" file that I create in one extra step for safety.
To create that file I extract the last MB of the future size.
Future size is 2201600 MB so I create the cut.bin with this command:
dd if=name-flat.vmdk of=cut.bin bs=1M count=1 skip=$((2201600 - 1))
So now we have everything we need.
The final command now is this one:
dd if=cut.bin of=name-flat.vmdk bs=1M count=1 seek=$((2201600 - 1)) conv=notrunc
Now this command re-injects our premade cut.bin of 1MB at exactly the same offset from where we extracted it in the first place.
This command will not yet cut the vmdk - instead it does actually nothing visible - it only reinjects and does not change the size of name-flat.vmdk
Last chance to doublecheck your value is NOW !!!! if you are sure run

dd if=cut.bin of=name-flat.vmdk bs=1M count=1 seek=$((2201600 - 1))

If you miscalculated your values or forget to set bs to 1MB you just probably destroyed your flat.vmdk.

If your values were ok - you just cut your flat.vmdk to the new desired size in place in just one second.

DISCLAIMER: this command is destructive - even I may not be able to help you if you messed this up.
DO NOT USE THIS IF YOU DO NOT UNDERSTAND THE INSTRUCTIONS !!!!

If you are unsure about the parameters post your vmdk descriptorfile and ls -lah from inside the VM-directory.
Ulli

ESXI doc

https://www.vionblog.com/migrate-virtualbox-vmware-esxi-6-5/

 Migrate VirtualBox to VMware ESXi 6.5

Using vmkfstools to clone the clone you will have 2 files Windows2008.vmdk (meta file that you can open and edit) and Windows2008-flat.vmdk (the actual data file)

To get the desired files use this commands:

cd /vmfs/volumes/volume01/yourmachine
vmkfstools -i Windows2008-tmp.vmdk  -d thin Windows2008.vmdk

-------------------------------------------------------------------

you can also limit the number of old logs that are retained by editing the VM advanced setting "vmx.log.keepOld"

vmx.scoreboard.enabled = "FALSE"

See volvo.vmdk

To set this configuration utilize the vSphere interface as follows:

Select the VM then select Actions followed by Edit Settings.

Click on the VM Options tab then expand Advanced.

Click on EDIT CONFIGURATION.

Click on ADD CONFIGURATION PARAMS then input log.keepOld with a value of 10.

Click OK, then OK again.

To set the number of log files to be used to 10, run the following PowerCLI command:

# Add the setting to all VMs

Get-VM | New-AdvancedSetting -Name 'log.keepOld' -value '10'

- Reclaiming disk space from thin provisioned VMDK files on ESXi (2136514) https://kb.vmware.com/s/article/2136514

Open an elevated command prompt.

Run this command

sdelete.exe -z drive_letter:

 Run this command:

vmkfstools -K disk_name.vmdk

------

Cannot login to Raspberry (looping)

Insert the microSD card to some linux PC and clean some files. Lack of free drive space might cause this

 sudo apt-get update

sudo apt-get upgrade
sudo apt-get dist-upgrade
reboot

sudo apt-get purge xinit

sudo apt-get purge lxde-core lxterminal lxappearance

sudo apt-get purge lightdm
reboot 

sudo apt-get install xinit
Enter the following command to install LXDE. This is the big bit. The download is 71 megabytes and it uses 242 megabytes on disk.
sudo apt-get install lxde-core lxterminal lxappearance
You now have time to use the boiled water to make and drink that coffee.
Enter the following command to install the LightDM login manager. The download is 25 megabytes and it uses 35.1 megabytes on disk.
sudo apt-get install lightdm

https://petermoulding.com/raspberry_pi/raspbian_lite_-_install_and_configure_with_lxde
https://www.tomshardware.com/how-to/fix-cannot-currently-show-desktop-error-raspberry-pi#:~:text=Boot%20to%20Desktop%20Mode&text=Select%20Boot%20Options%2D%3EDesktop%20%2F,Finish%20to%20exit%20the%20software.

Bluetooth is NOT in the panel of Raspberry PI

Open Terminal and type sudo apt-get install bluetooth bluez blueman. 

Press enter. https://www.cnet.com/how-to/how-to-setup-bluetooth-on-a-raspberry-pi-3/ 

d

https://www.vultr.com/docs/how-to-install-mosquitto-mqtt-broker-server-on-ubuntu-16-04

https://www.vultr.com/docs/how-to-install-mosquitto-mqtt-broker-server-on-ubuntu-16-04

How to Install Mosquitto MQTT Broker/Server on Ubuntu 16.04

Last Updated: Mon, Jan 15, 2018 

Linux GuidesPopularServer AppsUbuntu

MQTT is a publish/subscribe model based, "lightweight" messaging protocol over TCP/IP for communication between "Internet of Things" devices such as ESP8266, Raspberry Pi, etc. It is very popular with low resources and battery powered applications such as home automation, security alarm systems and battery-powered sensor networks.

Mosquitto is an open source message broker (or server) that implements MQTT protocols. With its good community support, documentation, and ease of installation it has become one of the most popular MQTT brokers.

Prerequisites

• An Ubuntu 16.04 server with root access
• Open port TCP:1883 on firewall

Step One: Install Mosquitto Broker

Update Ubuntu's package list and install the latest Mosquitto Broker available from it

sudo apt-get update
sudo apt-get install mosquitto

The Mosquitto service will start after installation.

Step Two: Install the Clients and Test

Install MQTT clients

sudo apt-get install mosquitto-clients

Mosquitto clients help us easily test MQTT through a command line utility. We will use two command windows, one to subscribe to a topic named "test" and one to publish a message to it.

Topics are labels used by the broker to filter messages for each connected client. A client program subscribed to a topic "Home1/BedroomTemp" will only listen to messages published to the same topic by other clients.

Subscribe to topic "test"

mosquitto_sub -t "test"

Mosquito_sub is a subscribe client we installed in the previous command. Here we are specifying "-t" followed by a topic name.

Publish a message to topic "test"

Login to the terminal as a second instance and publish a message to the "test" topic.

mosquitto_pub -m "message from mosquitto_pub client" -t "test"

Here the additional parameter "–m" is followed by the message we want to publish. Hit "Enter" and you should see a message from mosquitto_pub client displayed in other terminal where mosquito_sub client is running.

Step Three: Secure with a Password

Mosquitto comes with a password file generating utility called mosquitto_passwd.

sudo mosquitto_passwd -c /etc/mosquitto/passwd dave
Password: password

Create a configuration file for Mosquitto pointing to the password file we have just created.

sudo nano /etc/mosquitto/conf.d/default.conf

This will open an empty file. Paste the following into it.

allow_anonymous false
password_file /etc/mosquitto/passwd

Save and exit the text editor with "Ctrl+O", "Enter" and "Ctrl+X".

Now restart Mosquitto server and test our changes.

sudo systemctl restart mosquitto

In the subscribe client window, press "Ctrl+C" to exit the subscribe client and restart it with following command.

mosquitto_sub -t "test" -u "dave" -P "password"

Note the capital -P here.

In the publish client window, try to publish a message without a password.

mosquitto_pub -t "test" -m "message from mosquitto_pub client"

The message will be rejected with following error message.

Connection Refused: not authorised.
Error: The connection was refused.

Now publish a message with the username and password.

mosquitto_pub -t "test" -m "message from mosquitto_pub client" -u "dave" -P "password"

Hit "Enter" and you will see the message in subscribe client window, as in Step Two.

Conclusion

We have now set up a password protected MQTT server. You can use the Public IP of your Ubuntu server as an MQTT broker for your projects.

Want to contribute?