Linux, Red Hat, RHV, XenDesktop

Using Citrix PVS to stream Linux VDA (RHEL 7 Workstation)

I’ve been waiting a while to see this day and a sneaky section of the Citrix XenDesktop/PVS 7.12 notes indicate not only a lot more stability for the VDA Agent on RHEL but also that PVS supports (again) Linux targets!

If you’re not familiar with RHEL 7.x (7.3 in this case) you likely can use the same methods with CentOS 7 and Fedora.  If you don’t have a RHEL subscription, I would urge you to use CentOS 7, however, you can get a RHEL 7 subscription on the Developer portal (register here) (note it may just say 7.2 but once you run yum update and reboot it’ll be 7.3).

Let’s set the stage with what we are doing here.  We will be installing Red Hat Enterprise Linux (RHEL) Workstation 7.3 and configuring it to be a VDI Desktop in Citrix XenDesktop 7.12.  We are also using PVS 7.12 to stream a gold image to a VM that PXE boots and uses a cache disk (I choose to use cache to RAM overflow to disk).  Everything below is done as root for now. If you don’t like that, feel free to do it your own way but this is just a workstation we’re talking about here.

Install RHEL 7 Workstation

First thing you should do is install a VM, I’m using RHV4 and my VM consists of 2GB Memory, 1 vCPU and a 20GB disk.  The installation follows the guidelines in the documentation.  While their is an easy install option, we want to use the recommended best practice from Red Hat and leverage SSSD rather than samba.

First thing to notice is you need to scroll down to the section entitled “Prepare RHEL7/CentOS 7 for VDA Installation” which is not the first one (that is RHEL 6). Make sure you install Workstation and subscribe to get updates.  If you’re using RHEL you’ll need to run something similar to this

subscription-manager register
subscription-manager attach --pool={POOLID}
subscription-manager repos --disable=*
subscription-manager repos --enable=rhel-7-workstation-rpms \
                           --enable=rhel-7-workstation-extras-rpms \
                           --enable=rhel-7-workstation-optional-rpms \
yum clean all
yum install screen wget vim bash-completion git
yum update -y
#reboot if kernel is updated
systemctl reboot
#for ovirt/rhv I will install guest tools
yum install -y rhev-guest-agent
#if there is a bridge reported you might need to remove it before proceeding
#this would show up as some 192.x.x.x address in addition to your desired IP
#that wont play well with PVS and the VDA generally
#if that happens you can remove them with
#virsh net-destroy default
#virsh net-undefine default
#ifconfig #shouldnt have a virbr0 anymore

Follow the guide for setting the hostname and check hostname and hostname -f return the right values.  The command below enforces the hostname immediately.

hostnamectl set-hostname myhostname

Now modify chrony.conf for your ntp servers. You can use the following if you wish to run this in bash. Replace the IP addresses with the correct ones. (make sure you scroll to the right as the lines are long if you’re copy/pasting here)

sed -i 's/server iburst/#server iburst/g' /etc/chrony.conf
sed -i 's/server iburst/#server iburst/g' /etc/chrony.conf
sed -i 's/server iburst/server iburst/g' /etc/chrony.conf
sed -i 's/server iburst/server iburst/g' /etc/chrony.conf
systemctl restart chronyd

For JAVA, this is strange but if you use the default RHEL7 install the docs won’t work, that’s because they don’t have a symlink where Citrix says there is.  You should have java already installed if you’re using RHEL. I would recommend installing 1.8.0-openjdk and don’t remove the 1.7.0 if you’re using RHEL 7.2+ or CentOS7.  The path you want is /etc/alternatives/java. Let’s set it and source it in .bashrc (this will now always be a variable when you log in).

echo "export JAVA_HOME=/etc/alternatives/java" >> ~/.bashrc
. ~/.bashrc

Installing postgres should be simple, you can initialize without fear before we seal/clone so let’s do that.

yum install postgresql-server postgresql-jdbc -y
postgresql-setup initdb

systemctl start postgresql
systemctl enable postgresql

At this point you are ready for stage 2, the AD prep.  If you’re using XS or ESXi read the appropriate sections in the document.  I use Red Hat Virtualization because I like to live dangerously (and they also are my employer).

Active Directory Join (well not really)

This part can actually be done entirely and then we can remove the VM from the domain and clear credentials.  This isn’t a bad thing and if you do this, you’ll just want to leave the domain and delete the /etc/krb5.keytab (plus a few other things).  However we are going to do everything but join the domain at this point and leave that for later.

I HIGHLY SUGGEST you take a snapshot at this point.  If you love repeating steps (like the ones above, then ignore me but more than likely you’re going to mistype something here and need to revert).

Winbind sucks, so skip that part. We’re going with the latest, greatest SSSD.  My colleague, Dmitri Pal, discusses the best practice here if you’re a diehard winbind junkie.

Let’s run authconfig but also, we can specify all our variables up front now so you can just copy and paste and get on with life.  You can download them from this github and modify. Just follow the commands below

git clone
cd citrixpvslinux
vim #edit the file changing the options
chown root:root
chmod 600
. ./

If you type export and hit enter you should see the variables loaded for you.  We only want them for configuration so we load them now by sourcing the file (the . command above) and we will again after we create the gold image.

Let’s run our authconfig command now and modify the /etc/samba/smb.conf. You can run the github script or follow along. We aren’t joining the domain just yet though.

authconfig --smbsecurity=ads --smbworkgroup=${ADJOIN_SHORT} --smbrealm=${ADJOIN_REALM} --krb5realm=${ADJOIN_REALM} --krb5kdc=${ADJOIN_DC} --update
sed -i '/#--authconfig--end-line--/a kerberos method = secrets and keytab' /etc/samba/smb.conf

We will now prep the image a bit more for the PVS section, here we are writing in a file using our variables. You should check the file to be sure it looks right

cat >/etc/sssd/sssd.conf <

Now we are going to create a service that will join the machine to AD at boot time. We have to store the password in that file so we secure it, like we did above, with only root access.  This means you MUST be root to run this and access the file. Since we are going to join the machine with the command I HIGHLY suggest a minimum privileged account for this.  This guy seems to have a good walkthrough if you need one.  I also suggest you make an OU for the machines and add it as I have (it’s a variable in the export).  If you don’t do this, you’ll need to make the variable “Computers” or modify the command to join.

I suggest you use the github script and not copy and paste below.


cat >/etc/systemd/system/ctxvdaoneshot.service < /usr/local/sbin/ctxvdaoneshot
chmod 700 /usr/local/sbin/ctxvdaoneshot

cat >>/usr/local/sbin/ctxvdaoneshot <

To use this we would actually run ‘systemctl enable ctxvdaoneshot.service’ but we’re not quite ready yet.

At this point we’re as far as we’re going to get on AD without actually joining the machine. So let’s fix some issues that are going to pop up (I didn’t see any notes on this but you likely need to make these changes, otherwise your users will get some warnings)

Fixing the Display Issues

Run the following to fix the popups that occur if someone has a printer or moves the screen.  Most guest tools leverage VNC and this will prompt some security warnings (you can hit cancel and ignore them) about the color management and one about proxy for updates.  These are annoying and not easily fixed.  So the steps below will fix these issues by ignoring those warnings for anyone in the Domain Users group. Github script here.

(Update: Citrix had a ctx article on this now –

sed -e '$aX-GNOME-Autostart-enabled=false' -e '/X-GNOME-Autostart-enabled/d' -i.bak /etc/xdg/autostart/gnome-software-service.desktop

sed -i 's/Exec=\/usr\/bin\/vmware-user-suid-wrapper/#Exec=\/usr\/bin\/vmware-user-suid-wrapper/g' /etc/xdg/autostart/vmware-user.desktop

cat >/etc/polkit-1/rules.d/02-allow-colord.rules <

If you have GRID cards then good for you.  I don’t so I’m not going over that part.  However, if someone were to donate some to me I’d be happy to add that part, straight up trade.

Install the VDA Agent

Ok now we need to install this agent.  You need to actually download the rpm and put it somewhere. I use a local http server but you can just put it in the root directory.  scp works great for this. If you’re on a mac the command is something like:

scp ~/Downloads/Citrix/XenDesktopVDA- root@linuxvdafqdn:/root
# prompts for password of root and it will copy
# lets also copy the PVS agent (it is on the ISO for PVS 7.12)
scp ~/Downloads/Citrix/pvs_RED_HAT_7.12.0_BETA5_9201_x86_64.rpm root@linuxvdafqdn:/root

You should now have two rpm files in your ~ or /root directory. I guess you could use Citrix’s commands in the doc but I don’t do that. I generally always use the following for RPMs I find.  You can also use this for an rpm on a http site (just replace the location with the web address).

# install prerequisites if not already installed (it should warn you if not)
yum install -y ImageMagick motif foomatic-filters
rpm -Uvh ~/XenDesktopVDA-
systemctl enable ctxhdx ctxvda

At this point we’ve completed the VDA setup without having joined the machine to the domain.  A reboot and snapshot is advisable at this point.

systemctl poweroff
# take a snapshot
# bring it back up
# source variables
cd ~/citrixpvslinux
. ./

PVS Install

You should have the rpm in the /root directory or somewhere. Let’s go ahead and install it along with a prerequisite.

yum install -y tdb-tools
rpm -Uvh ~/pvs_RED_HAT_7.12.0_BETA5_9201_x86_64.rpm

The following commands are in this github script but you can follow along. We need to set selinux to permissive. This kind of sucks but another day I’ll work through that.  What you don’t want to do is disable selinux, you should NEVER be doing that honestly unless you’re exploring exploits or somehow time travelled prior to Y2K (buy some stock while you’re there!). Anyway let’s get this done.  I’m assuming selinux contexts aren’t transcribed properly when we stream the OS disk.

# Set to permissive for the immediate timeframe
setenforce Permissive
# Set it to persist on reboot (if you dont do this it will revert to enforced)
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/sysconfig/selinux

We also want to add our CA Trust because PVS runs securely now and the linux target need to trust the issuer.  You could use the option for self-signed in the citrix docs, but I’m assuming you’re serious about this and have a CA somewhere you can spin up a cert for.  You can grab the cert from the CA by using this command (it’s also part of the script).

openssl s_client -showcerts -connect ${ADJOIN_CA}:443 /dev/null|openssl x509 -outform PEM > /etc/pki/ca-trust/anchors/${ADJOIN_CA}.pem
update-ca-trust extract

Now let’s add a Cache Disk for PVS to use, I set mine up for 10G and created it without rebooting or shutting down.  My main disk is /dev/vda and the new disk is /dev/vdb.  If you used my script from github at this point, just ensure the variables are right and run it or run the following.

#export TGTDEV=/dev/vdb
sed -e 's/\s*\([\+0-9a-zA-Z]*\).*/\1/' << EOF | fdisk $TGTDEV
 o # clear the in memory partition table n # new partition p # primary partition 1 # partition number 1 # default - start at beginning of disk # default, extend partition to end of disk p # print the in-memory partition table w # write the partition table q # and we're done EOF
mkfs.xfs -L 'PVS_Cache' ${TGTDEV}1

After this it’s time to seal up and convert our VM to a PVS image.  You’ll want to remove the subscription-manager info also to clear that for use but you can leave the hostnames alone since PVS will change them for you.  The github script for this part is here.

Let’s cleanup yum and also enable the vdaoneshot script too.  We assume you have the collection, store name and all that info correctly inputted to the or export.

yum clean all
subscription-manager unregister
subscription-manager remove --all

systemctl enable ctxvdaoneshot

# Lets create this

echo "If something went wrong please run: systemctl disable ctxvdaoneshot"
echo "before rebooting...."
echo "If you succeeded, please clone this image and remove the OS disk"
echo "Then create a template, create the VMs and add them to the collection in PVS"
echo "DO NOT FORGOT to set the boot order to use the NIC first"
echo "You also should have options 66/67 set in DHCP (they are the same old options from years ago)"

At this point you should be creating your image.  You’ll want to clone, remove the OS disk and then make additional VMs based on that VM. I had to manually create the additional VMs in PVS, transcribing the MAC addresses but it works.  This method creates Desktop VDI machines so select that when you create the machine catalog and delivery groups in XenDesktop.


Once all is complete you should be able to see your images and launch a desktop!

EDIT (12/22/16): I wanted to also mention the setup on the PVS side and 😄 side.

I am a big fan of stream to RAM but the notes on the Citrix site are a bit cryptic on the setting.  I understand them to say that Linux uses the RAM for cache that is there (Linux manages this) so the cache maximum is just a reserve and you want this set low to 8MB which is what I did.  I also have a disk setup as overflow.  It is vital you name this PVS_Cache in the mkfs.xfs command. I didn’t try lvm because I didn’t think it would work, I suggest you stick to xfs or ext4.  I love btrfs but unless you know a LOT about that filesystem I wouldn’t use it for default (the benefit of btrfs is generally the ability to add/remove disks from RAID without impact, kind of like lvm but lower level).


On the XenDesktop side, you can use the shared mode but I never had a lot of luck with the desktop there (Citrix said it should work so I’ll try later). I used the VDI mode for now.


Career, Uncategorized

What would you say you do here?


When people ask me what I do it always puts me on defense trying to figure out how much knowledge you could handle and then speaking within a few seconds.  I heard a colleague, John Willis, speak and he discussed the standard response of “computers” which is a solid answer to avoid a long discussion.  Even better, a former colleague, who was an amazing saleswoman at EMC, used to say “I sell bananas!” when it was clear there was nothing happening.

I’ve thought about this a lot and kept thinking back to Officespace with the Bobs asking Tom Smykowski the question above.  Tom responds with a painfully obvious “nothing useful” that he seems to realize as he says it.  Basically he brings the specs to the engineers from the customers so the engineers don’t have to. When the Bobs ask why can’t the engineers bring them to the customers themselves, Tom shouts that he is a people person, gets flustered and exhorts, “What is wrong with you people!”.

While I don’t think my job or most others falls into this, it’s important to be able to describe what you do.  There is a saying that if you truly understand a subject, you can explain it to anyone, even a child.  I believe this to be true and I think it’s important that those in IT be able to understand their job and be able to explain it.

If you’re wondering, I have tried explaining what I do to my kids and wife with varying degrees of success.  I’d also encourage you to do the same.  If you get flustered or worried, I think you have every right to circle back to your manager or senior level company person and get some answers.  If they don’t know, take advantage of that open-door policy the CIO or CEO always brags about.

In DevOps, I always stress Value Stream Mapping.  This is a process where we take a workflow that involves our business (like a customer ordering a product and us making it) and we break it up into boxes, visually, and look at cycle times, delays and where we are in the process.  It’s important to do this visually because we don’t always see the big picture clearly when we are on the floor or entrenched in work.  It’s also invaluable because it shows where you add value (or where you don’t) and gives you the opportunity to change that.  You can see that when I do this, I do not add any value to the overall goal, or you may find that you are adding tremendous value and not realizing it.  In any case, the goal is to understand how your company works, how it makes money and where you are in the process.  If you take this and apply it to IT, you should find that your role is a part of the business and does add value.  If not, or if no one knows how your company works, you should start asking questions and looking for better opportunities.

Today I work at Red Hat. Red Hat takes community projects and turns them into enterprise products.  If that’s hard to grasp we take free software and add support and stability for companies to use and be able to trust.  I get paid based on the sales made from the 2 salesmen I support.  Our products help businesses focus more on their business and less on IT.

Next time you want to say computers, try to explain it.  If you fail, you’ll get another shot.  However I think in time, you’ll either learn to explain what you do to anyone or realize you can do something a lot more worthwhile.


Citrix, Uncategorized

Citrix Synergy as CTP

Getting to attend Synergy 2016 is one of the perks that accompanies being a CTP.  You also have responsibilities that come with this but I’m finding the perks seem to outweigh them!

I joined Red Hat last week and I think I’m the only Red Hatter here (I get weird looks with my hat but that’s on purpose).  Citrix is heavily aligned with Microsoft, Microsoft announced Red Hat on Azure and there has always been interest in VDI with Linux.  That being said, I get to come to Synergy as more an outsider than ever before (instead of from the customer or partner side).

CTPs have quite a few meetings lined up when they come to Synergy that start 2 days earlier than the kickoff.  These meetings are no joke and I got plenty of warnings about pacing myself.  They started at 8am promptly and continued to about 7pm two days in a row with few breaks.  The topics are intense and interesting and it’s easy to get overwhelmed.

The meetings are generally with the Citrix product teams and we get to learn and give feedback to new releases, ideas, successes and failures.  Each team is different in terms of how they operate.  I can’t go into too much detail but Citrix is very interested in improving their feedback loops and ensuring that products developed have purpose instead of just throwing things over a wall and surprising people.  This is a good move as it allows a lot more trail and error, and a lot less of “what the f is this thing?” when something in announced.  This is definitely the primary interest I had with my CTP meetings, in addition to the awesome company!

This year I’m not speaking at a breakout but doing some other activities.  I’ll be at the Synergy Showdown #XAonAzure with my team, the Village Idiots, along with Esther, Jarian and Paul from the CTP team.  We will be taking on the wily dutch team but our secret weapon hails from the Netherlands (Esther) so we are a lock in to win 😉 !  I will also have a few match.geek sessions if you want to hear me blather on about whatever you want and I’m hosting the education tech chat tables (except for today, Tobias will be doing Tuesday’s due to the showdown).

Today was fun, I got to sit at the bloggers table and get super secret early access the the keynote.  Esther even got me a front row table seat and the view is spectacular.  If you don’t think a CTP has perks, you’re mistaken and this is one of the best perks I’ve ever had.

If you’re here and want to connect, let me know. You can look for a dude wearing a red hat (literally) and it’s likely me or DM on twitter or the Synergy app. Looking forward to the rest of the week and running into old friend and making new ones!

Career, Red Hat, Uncategorized

First week at Red Hat

I recently joined Red Hat on May 16th and spent a few days in Raleigh at orientation.  I can’t think of a time before that I felt as good a fit as at Red Hat.  Don’t get me wrong, this isn’t a job to sit on your ass, however, if you work hard, you are rewarded.  The company culture seeps out of people and as best I can describe it, it is basically one of common sense.  Nothing overly crazy, but a lot of thought put into decisions, then once made, they are implemented quickly.

For any of you wondering why I made this move, I had spent the last year at Ericsson working on a very large OpenStack rollout with AT&T.  What we did was great but, my heart lies with presales and I couldn’t pass this opportunity up.  I’ve always tinkered and been a hobbyist with Linux and love the thought of having a career with it.

I have no idea how much I can weld my past technologies with what I am doing today but time will tell and you have to make changes in life.  Nevertheless, I am at Citrix Synergy next week in Vegas, so if you’re out there let me know and we can meet.

OpenStack, Uncategorized

OpenStack Summit – keynotes

Thanks to Champions Solutions Group, I’ve been able to attend the OpenStack Summit this year, 2016, in Austin, TX.  The event is interesting considering my experience with other conferences.  Generally I attend those that are geared towards partners, customers and sales. However, OpenStack is geared to a wide audience, but especially towards the engineers, architects and users which makes me feel right at home.

Highlights included the attendence of 7500 people and also the revelation that 65% of deployments of OpenStack are now production deployments!

OpenStack’s keynotes focused on a few themes.  Donna Scott from Gartner was correct in pointing out “disruption” as a key motivator explaining bimodal methods of deployment (mode 1 being reliable stable and mode 2 more experimental but agile). AT&T presented their challenges and wowed the audience with the rate of growth of mobile data (150,000% from 2007-2015!).  They are adopting a white box solution to lower cost, increase speed and agility. They also talked about a management layer for mult-tenant, complex networks.

Mirantis had a great presentation and poked at Gartner on their past of saying OpenStack wasn’t a real player (but that it’s in the past and happy they are hear).  After some story about Russian and vodka drinking bears, Mirantis made some excellent points.  Namely, people and process over tools and technology.  This is exactly the core of DevOps and if you know me, is a good overall view (I wish they went into value stream mapping a bit but it’s a start!).  They also pointed out that while they are a big AWS fan, there is ENORMOUS market share outside AWS and AWS has a teeny part of it!  They also poked a little fun at the vmware guys (not all, I know many are embracing change) but that adopting agile and openstack without looking at people is a killer.  Namely public cloud adoption seems great because you don’t have to deal with the people, but it’s not cheaper or always better.

We also heard from SAP and VW and Red Hat.  Red Hat stayed on message with the 65% of deployments to production.  SAP explained their addressing the movement of the industry with a customer expecting instant response time with a tweet. VW explained the connected car and the use of cloud computing.

Day 2 had keynotes that focused on containerization.  QUite a lineup of presenters including Cisco, OVH, TWC and liveperson and smartcities (and more) spoke. I think the best highlight was the presentation by Google and CoreOS highlighting tectonic which overlays Kubernetes (  I may have got this wrong, but each kubernetes node can hold many containers and they run openstack components as containers. They demonstrated a horizonv2 to v3 upgrade and also self-healing by killing a neutron and then a nova node.  I was definitely impressed and need to shore up by kubernetes skills!


Citrix, PVS, Uncategorized

Citrix PVS and Managed Service Accounts gMSA

I’m a big fan of Managed Service Accounts because they are much more secure and aren’t easily exploited by human beings.  Basically, Active Directory controls the account with it being responsible for changing passwords.  While use of gMSA (group managed service accounts) is sometimes hit or miss, I didn’t find much on recent use with Citrix other than a vague “we support this” statement.

Carl Webster had a much older attempt with PVS (not sure whether he tried again or not) and I wanted to ensure that this worked on PVS 7.7 (just released).

You’ll need a couple of things

I would leverage a tool for creating and managing gMSA that I got here.
(note: for a quick guide on setting this up, I would look through Derek Seaman’s blog).


Add you PVS server to the list otherwise it won’t work. (I only have 1 PVS server right now, I’m in rebuilding mode…)


For SOAP, you’ll need to make this account a member of the local admins on the PVS server (when you add the account, make sure you select “service accounts” for objects.


For SQL, I am using 2014 with availability groups.  Check out Derek’s blog for a great walkthrough on this.

Your database should have been created already (use the dbscript.exe to manually create the database in PVS)

Grant the permissions needed to your gMSA on the SQL database (I create the account on both database servers just in case (when I test the failover))

Testing failover should work and you will also notice the services are runningpvsconsoleservicespvs

Citrix, XenServer

FusionIO and XenServer

Getting FusionIO cards and XenServer to work seem to be a bit tougher than I thought but it’s possible. I have a few cards, some old servers and a XenServer 6.5 ISO.  I was hoping to use 6.6 but there isn’t a DDK up for that yet.

Install DDK 6.5
Increase the memory (I used 4GB) or you’ll run into errors on the RPM rebuild with out of memory

on XenServer 6.5

install lsof
yum install –enablerepo=base lsof

you’ll want to use the DDK to develop an RPM as specified here –
Now you can download the HP stuff here – I wouldn’t I’d use the sources from FusionIO directly.  Download the Centos5 sources and all the Utilities from here –

Use WinSCP to upload the files to your DDK server
You should upload the Sources directory

run rpm –rebuild iomemory-vsl-*.src.rpm

Copy the rpm in the /usr/src/redhat/RPMS/x86_64 directory on the DDK to the XenServer
Also copy the utilities directory from the downloaded ZIP to the XenServer

rpm -Uvh iomemory-vsl-*.rpm (you may need to do this in order)

You can ignore the dracut error (search for it in the pdf here)

cd to Utilities – run the following (check the order I may be wrong here but it’ll tell you)
rpm -Uvh fio-common*.rpm
rpm -Uvh fio-preinstall*.rpm
rpm -Uvh fio-util*.rpm
rpm -Uvh fio-sysvinit*.rpm

now run


Note the driver version is empty? – now we are in sync with Kyrian’s blog post.

Run fio-status again – I had to upgrade the firmware on mine meaning I had to copy the firmware file *.fff to the XenServer and run


Then reboot and fio-status again. Now I have to wait for some low-level format but that’s ok…


I and listed what I had for /dev/fct* and performed the following command
fio-format /dev/fct0
fio-format /dev/fct1

Then attached them
fio-attach /dev/fct0
fio-attach /dev/fct1

After that fio-status looks good and we mount
xe sr-create name-label=”ioDrive00″ physical-size=320Gb type=lvm device-config:device=/dev/fioa
xe sr-create name-label=”ioDrive01″ physical-size=320Gb type=lvm device-config:device=/dev/fiob

And we are done!