Background
Our customer had an existing data acquisition system that suffered from the following data management issues:
- Local data storage
- No backups were taken, so data was at risk
- The data was not secure and there were critical security vulnerabilities
- The database grew unmanaged over time, so it suffered from slow data write and export speeds
- Off-the-shelf data acquisition software could not support the data throughput and acquisition rates required by changing industry requirements
- Operators were bottlenecked in their daily tasks due to the single user-access point on the one PC hosting the local data
- The database offered limited data management tools, so getting insights from the data was difficult
- Data was aliased according to the physical channel it was acquired on, so handwritten notes were needed to associate a physical channel with the meaningful metadata of the test
- Data was cumbersome to visualize and response rates for the interface were very slow
DMC's Approach
In an effort to resolve these problems, our customer sought to find a solution that could gather high-speed test data from their fleet of remote NI cDAQ devices and store the data in a secure and centralized location. The customer also wanted a solution to accommodate multi-user remote access and provide tools for organizing, searching, analyzing, and reporting the data. Based on these requirements, DMC designed a solution that leverages SystemLink’s wide range of data management tools to accomplish our customer’s goals.
Step 1. Test Executive App
DMC specializes in custom LabVIEW applications and we worked closely with the client to develop a test executive that would fit well into their existing testing workflow. DMC began by designing a flexible test executive for configuring and controlling the data acquisition hardware. This was a LabVIEW application that ran on local computers (nodes) connected to the data acquisition devices. These “nodes” provide a local interface to the data and allow operators to configure hardware settings, configure data acquisition parameters, start new tests, and view live results in high resolution.
In order to send data from the “nodes” up to the SystemLink server to leverage the different features that it provides, DMC developed a code module called the Data Destination. The Data Destination is an object-oriented class that was designed to be a generic and flexible API for the test executive to store acquired data in different locations. SystemLink data destinations were implemented to allow the test executive to automatically send data up to the SystemLink server in the form of both data files (for high-speed data) and SystemLink tags (for calculated data and metrics).
These drivers will be discussed in further detail in the next section. Other potentially useful Data Destinations could include a separate Database destination or even a Live Data Plotting destination.
Step 2. SystemLink Data Destinations
SystemLink has two great options for data storage, files and tags.
The file storage on the SystemLink server is enhanced and optimized by the SystemLink software. The server provides a centralized and secure option for storing data files and the SystemLink File Transfer toolkit can be used to programmatically transfer files from a custom LabVIEW application to the server. DMC developed a SystemLink TDMS driver based on this API to save and format high-speed test data to TDMS files and transfer them automatically to our customer’s SystemLink server during testing. The API relies on HTTPS and SystemLink’s system management feature to validate and securely transfer files.
The SystemLink tag feature is useful for storing slower speed data such as metrics and health data. The Tag Viewer in the SystemLink web UI makes it easy to view the latest tag data as it is being updated live and the tags are great for monitoring and tracking ongoing tests. The “Aggregate” feature enables historical recording of tags values that can be viewed through the tag viewer as well. Features such as keywords and properties allow the user to attach metadata to the tags for easier organization and filtering during searches. The SystemLink Tags toolkit provides the tools necessary to create, configure, and interact with SystemLink tags in the LabVIEW environment. For this project, we created the SystemLink Tag driver to calculate test metrics and post them periodically to SystemLink tags for general test monitoring. Tags were created dynamically from the custom LabVIEW application so that by the time they were published to the SystemLink server they were ready to be consumed by end-users, who were looking to understand their tests.
Step 3. SystemLink Server Setup
An important consideration of integrating SystemLink into any project is the specification of the server itself. The performance of the SystemLink server is dependent on the CPU, RAM, and drive that is selected for the server hardware. Data throughput, the duration of tests, and retention periods of data should be considered to calculate how much disk space the SystemLink server would need to accommodate the data that is collected in the lifetime of a system.
DMC worked closely with our client to calculate the hardware specifications needed for a high performing, data storage solution. DMC also benchmarked the SystemLink file transfer and tag writing features using a SystemLink test server to ensure that the application would meet or exceed the customer’s requirements. The final proposed server hardware was designed to last for years of testing and high-speed data collection.
Step 4. SystemLink Features
Once test data is stored on the SystemLink server in the form of data files and tags, SystemLink’s wide range of data management tools become available to the user.
The SystemLink web interface allows simultaneous multi-user access to the tools and stored data. The web interface can also be configured to require user authentication at different user levels for security purposes. The high-speed TDMS test data can be accessed and viewed directly in the web interface using the file viewer tool, which also provides additional filtering and data manipulation features. Tag data can be viewed through the tag viewer and can also be used to create test dashboards or queried by the Test Monitor to produce test insights.
SystemLink’s systems manager tool allowed DMC to set up asset management for our client’s test nodes. This unlocks the ability to push software updates to the nodes through the web interface and easily track metrics such as connection status and CPU usage. Calibration tracking can also be configured for managed assets to notify users based on calibration schedules. Overall, SystemLink’s asset management provides the user with the ability to ensure configuration conformity across assets and quickly view the status of an entire fleet of test systems all in one place.
Overall, DMC provided our client with a solution delivering the following features that resolved many of the issues with their previous data management system:
- Centralized data storage on a server with backup options
- Secure data transfer
- Improved data storage security
- Multiple data storage methods for both slow and high-speed data
- Simultaneous multi-user access
- Access to broad selection of data management and analysis tools
Conclusion
With the new test executive and SystemLink integration, DMC was able to deliver a fully-featured data acquisition and data management system to our client. SystemLink is an extremely powerful software suite that enhances the functionality of a data server and provides the user with a wide range of tools to effectively store and interact with test data. DMC’s SystemLink experience and status as an NI SystemLink Specialty Partner makes us well-positioned to help with the integration and configuration of SystemLink for a wide variety of test and measurement systems. We have the expertise to help our clients specify hardware, work closely with IT during server setup, and deploy SystemLink solutions.
If you are interested in implementing SystemLink in your own systems or want to learn more about its features, feel free to reach out to us!