Due to lifecycle management (LCM), we replaced several Citrix NetScaler appliances with new ones. Although we conducted thorough acceptance tests before putting them into production, unfortunately, we experienced an annoying issue once they were operational.

Some users complained that they saw a spinning progress bar after they successfully logged on to the Citrix NetScaler. It was only reported by a minority of users and was resolved by refreshing their web browser sessions. In the end, users stopped reporting the issue because it occurred infrequently and the solution was simpleā€”just press F5. We initiated an investigation in the hope of completely resolving the issue.

Read More →

A “security.txt” file is a standard proposed by security researcher Ed Foudil in 2017 as a way for websites to define a security policy. It’s akin to the well-known “robots.txt” file which specifies rules for web crawlers. The security.txt file allows website owners to provide information to security researchers about how to report security vulnerabilities or concerns.

Since April 2022, Security.txt has been an Internet Engineering Task Force (IETF) informational standard

Read More →

Recently, we worked on upgrading a Citrix NetScaler VPX from version 13.0 to the latest 14.1 build. The Citrix NetScaler VPX, which had been running for quite some time, had not been upgraded because it still used features and functionalities, including Classic Policies, which essentially needed to be replaced by Advanced Policies starting from the 13.1 build.

During the preparation for the upgrade, our main focus was on the legacy configuration in the running ns.conf file that needed to be adjusted.

Citrix ADC scripts for migrating and converting Citrix ADC configuration with deprecated features https://github.com/netscaler/ADC-scripts/tree/master

By using the NSPEPI tool, you can not only check for legacy configuration but also convert it to new configurations in many cases. Always ensure that you download and use the latest version during the analysis. If you are upgrading from a version older than build 13.1, always use NSPEPI beforehand to ensure that everything continues to work as expected after the upgrade.

check_invalid_config /nsconfig/ns.conf

After replacing all legacy configurations in the ns.conf and ensuring there were no blocking issues according to the NSPEPI tool to upgrade to the latest 14.1 build, we conducted a trial upgrade migration within our acceptance environment.

After the upgrade, the Citrix NetScaler restarted smoothly, but it was no longer possible to log in using our domain accounts (LDAPS). Fortunately, logging in with the local nsroot account still worked. Once logged in, it was immediately apparent that several load-balanced VIPs were down, causing the LDAPS load balancer to be inactive. Additionally, various NetScaler features were suddenly no longer visible.

Show Unlicensed Features

The navigation suddenly included an item labeled “Show Unlicensed Features,” which we hadn’t seen before. After clicking on it, all features became visible again. However, it became immediately apparent that many things seemed to be unlicensed all of a sudden. Features that we were using prior to the upgrade to build 14.1. While browsing through the NetScaler GUI, we navigated to System > License and discovered that we were running an Express edition instead of Platinum. Consequently, many of the commonly used features were indeed unlicensed.

ADC License

Next, we examined the existing license files located in the directory /nsconfig/license. What immediately caught our attention was the date present in the license file. In our case, the expiration date was older than the Eligibility Dates required for using the Citrix NetScaler 14.1 build, which is July 12, 2023 šŸ™

NetScaler License File

Citrix products and their Eligibility dates https://support.citrix.com/article/CTX111618/citrix-product-customer-success-services-eligibility-dates

Since this was a Citrix NetScaler VPX with a valid software subscription, the solution was fortunately quite simple. Simply redownload your license file via the MyCitrix license portal and upload it to the Citrix NetScaler VPX. The new license file will include a new SA Date, enabling you to run build 14.1. After restarting the Citrix NetScaler, all previously licensed features reappeared.

Check your product eligibility dates before you proceed with the upgrade!

In the past, it was possible to upload your NetScaler configuration file (ns.conf) to the Citrix Insight Service, which would then conduct an automated health check of the configuration. You would receive a report detailing any potential issues, best practices not followed, and so on. This was incredibly helpful during setup. Unfortunately, Citrix discontinued this self-diagnostic service some time ago.

During E2EVC 2022 Athens, I stumbled upon the “Arrow’s NetScaler config analyser” in one of the sessionsā€”a tool more than handy. After registration, it allows you to check your NetScaler configuration for free. However, in practice, I still regularly encounter NetScaler administrators who are unaware of its existence, so I thought I’d mention it again.

Arrow’s NetScaler config analyser https://app.xconfig.io

Although they offer more than just the free health check, in this case, I want to specifically mention the FREE “Online Config Analysis.”

Unless you choose to save your ns.config within your personal account, your ns.config is not uploaded to their website; instead, it is analyzed locally from your browser session.

For added security, it’s advisable to first mask any confidential data such as passwords, IP addresses, etc., ensuring they’re not usable.

Without registration, not all results are visible, so go ahead and register yourself.

After creating an account, you’ll have full visibility into all the issues discovered within your ns.conf. These issues are categorized into four categories:

  • Critical
  • Major
  • Medium
  • Low

If you ask me, your configuration shouldn’t contain any Critical, Major, or Medium findings! šŸ˜Š

An example of a Critical finding might be:

An example of a Medium finding might be:

What’s also very handy besides the analysis of your NetScaler configuration is the easy browsing through your configuration. By selecting an item on the left side (which looks identical in structure to a NetScaler), you’ll see the corresponding lines from your configuration on the right. This makes the configuration much more readable and understandable.

The tool is constantly evolving, with new recommendations being added regularly. For a comprehensive overview of the change log, you can navigate to the “What’s new” section. If you encounter false positives or have recommendations for improvements, don’t hesitate to let them know. In my experience, they are responsive to user feedback and often address issues or implement suggestions in subsequent releases!

When designing environments, we always think about the high availability of the different components. Two is one, one is none! So when designing a ShareFile environment I want at least two StorageZone Controllers (SZC) for every StorageZone. Because most Citrix environment already contain a NetScaler it’s my preferred method to also use the NetScaler for load balancing the StorageZone Controllers. Nowadays the Citrix NetScaler has some nice wizards build in to assistĀ you deploying the ShareFile configuration.

Two is one, one is none. Load balance the StorageZone Controllers!

The wizard however will use the “tcp-default” monitor to check the service state. This means the NetScaler appliance establishes a 3-way handshake with the monitor destination, and then closes the connection, to check if the destination is up. Although this is configured by the wizard and a best practice, I believe this is a weak spot in my HA setup. I have seen situations where the SZC server was running, but the ShareFile services were not responding correctly. A simple tcp monitor will not notice this!

The NetScaler ShareFile wizard uses the default tcp monitor

Read More →