Storing and uploading data
This article summarizes how Movere stores and uploads collected data.
Data collection
The table summarizes data collected by Movere.
| Data | Details |
|---|---|
| User data | User data is collected during the registration process. Review the personally identifiable information (PII) data collected during registration. |
| Movere Console data | The Movere Console is unique for each customer, and is identified by a GUID. The GUID doesn't include the customer name, user name, or any other PII elements that link it to a specific customer or user. The Console also includes a PGP public key, exclusive to a customer. The PGP key is 2048-bit, and is used to encrypt all data before it's uploaded to the cloud. Review the PII data collected during Console deployment. |
| Scanning data | All data collected during scanning is uploaded to the Movere cloud. Data can be uploaded automatically (directly from target device or via the Movere Console), or uploaded manually. Review the PII data collected during scanning. |
Data upload methods
Methods for uploading data to the Movere cloud are summarized in the table.
| Upload method | Details | Upload process |
|---|---|---|
| Upload automatically | This method is the default setting on the Uploading Scans page, in the Movere Console. | If you don't enable this option, you need to manually upload scanned data to Movere. |
| Upload directly from target devices | If you're uploading automatically, this option uploads data directly from scanned devices. The advantage of this method is that you can scan an environment without adjusting firewall or port settings. To use this option, the Customer Tenant user account needs Write access. The account authenticates, and downloads a token.txt file. This file contains the encrypted Movere user ID. |
Data payloads upload directly from the device to Movere on port 443 external outbound, using TLS 1.2. Movere bots try to upload data directly from the device three times. If three attempts fail, Movere tries to send the payload back to the Console over port 443 internally, so that it can be uploaded from the Console. If upload to Console also fails, then the Console pulls the data from the target machine, using the File and Printer sharing mechanism. |
| Upload to the Console | If you select to upload data directly from scanned devices, it's only sent back to the Console for uploading if three attempts to send data directly from the device fail. However, in some circumstances, you might want to specify that data should always be sent to Movere from the Console, and disable uploading directly from scanned devices. |
With this setting, all payloads are sent internally over port 443 to the Console device, using the TLS version (1.0, 1.1, or 1.2) that's supported on both the Console machine, and target devices. If upload to the Console fails, then the Console pulls the data from the target machine, using the File and Printer sharing mechanism. From the Console, Movere uploads data to the cloud, making three attempts (with a wait time of 60 seconds between each) to send payloads on port 443 external outbound, using TLS 1.2. If payload uploads fail, then the Console tries uploading after every 12 hours (from the start of the scan). |
| Upload if port 443 is blocked | This method is for inventory scans only. It's used if you specify that data should be sent directly from a device, and the device can't send data to Movere, and can't send it back to the Console over port 443. |
The Console waits two minutes from the start of the scan, and pulls the payload from the target device. - A message is recorded in the Log.Service file to indicate that the payload was pulled. - Movere doesn't record a "Local scan started" message in the log file. This message indicates that the target device can communicate with the Console. If you don't see this message, it's an indication that the target device probably can't communicate with the Console internally over port 443. |
| Upload manually from the Console | Use this method if you don't want to upload data automatically to Movere. Payloads are stored locally in the Console > Uploads folder. Each day scans run, a new subfolder is created with the current date-time (UTC) stamp. |
If there are scan files in the folder, you can select the Upload Scans option in the Console, and upload them manually. Note that resource consumption payloads are time sensitive, and must be uploaded within 24 hours after payload generation. If you upload after 24 hours, an upload success message is generated in the log files, but data in the payload isn't processed in Movere. |
| Upload using pull mechanism | Used for actual resource consumption scanning only. If the target machine can't upload payloads to the cloud, or to the Console, using the push mechanism, then the Console pulls payloads from the target machine every six hours, from the start of the scan. |
The Console pulls the payloads using File and Printer sharing. Movere must run as a Windows Service (start from Console, or use the -startlistener flag). UDP 138, UDP 137, TCP 139, TCP 445 must be open on the Console device (outbound), and on the target device (inbound), for the Console to pull payloads successfully. |
| Upload from Console to cloud | Payloads are uploaded to the Console. | The Console uploads the payloads to the cloud, using TLS 1.2. |
Processing uploaded data
Movere doesn't automatically process uploaded scan files as they arrive. Movere waits for the current scanning cycle to conclude, and processes all uploaded files as a batch.
| Type of Scan | Processing Cycle |
|---|---|
| Inventory Scan | The processing of inventory scan starts immediately after the last payload as been received. The payload is assumed to be last payload when atleast 3 minutes has elapsed from prior payload and no additional payload is being received. |
| Actual Resource Consumption Scan | The actual resource consumption scan payloads are processed once in a day on a daily basis. This processing starts at 12:00 AM UTC. If the resource consumption scan payload is received until midnight UTC, then the payloads are processed until 6:00 AM the following day and and the data is displayed in the portal. |
Upload security
Each Movere Console contains a unique, 2048-bit, strong PGP key specific to each Customer Tenant. It's referred to as the public key, and can only be used to encrypt data within a Movere payload.
- Each customer is issued a unique 2048-bit strong PGP key, which is used to encrypt data in memory before being written to disk, significantly reducing security risks. This is referred to as the public key, and can only be used to encrypt data.
- To decrypt the data, it must be uploaded to the cloud, where specialized APIs identify the user, match it to a customer, and retrieve the customer’s private key from a repository that stores it as encrypted.
- The user uploading data using the Movere Console need to have the correct access level (Write claim), and must authenticate using the Movere Console.
- After the user authenticates, they're issued an access token (token.txt).
- The token is valid for 12 hours, and is used for every upload for which the Movere Console is responsible, whether inventory or utilization data.
- After the access token expires, a new token is issued, that's also valid for 12 hours, and so on, until reaching the maximum time for actual resource consumption scanning (90 days).
- The access token stored in token.txt isn't used to encrypt or decrypt data, and can't be used to access the Movere web portal.
- The sole purpose of the token file is to allow uploads of already encrypted data to the cloud, and to identify the user performing the upload.
- After 90 days, no new tokens can be issued, and user needs to authenticate once again via the Movere Console
- After reaching the cloud, each payload is handled by a FileTransfer API. Each payload is decompressed and decrypted, before being transferred to the specific customer database.
- No two customers share the same database.
- The data is also extracted onto a secondary database for reporting, then into Qlik, which stores it in memory for the user to access it in the Movere portal. . No Customer Tenants share the same database.
- The entire upload process occurs over a secure connection (HTTPS) which uses TLS 1.2. This is in addition to the encryption at rest of each file, using PGP keys.
Data storage
Movere is a cloud solution, and stores all encrypted and anonymized data in the Microsoft Azure cloud. Movere is covered by the Azure platform security processes and certifications. Learn more.
- All data captured is stored in its original encrypted format, within the Movere Azure environment.
- The Movere infrastructure is currently configured in the Australia East, Canada Central, Central US, West Europe, UK South and West US 2 Azure regions. Each customer database resides in only one of these regions.
- Customer data is stored only in the Movere region in which the database resides. For example, if a Customer Tenant resides in Australia East, all scan data resides only in this region, including any backup data. No scan data is replicated or stored in other Movere regions, at any time.
- In the unlikely event of database or backup loss, the database can be recreated using the original files uploaded to Movere.
- User sign-on data is replicated across all Movere regions, to ensure the quickest authentication experience for users anywhere in the world, regardless of the user account region.
- For example, if a user registered to a tenant residing in West US 2 logs in to Movere during travel abroad, Movere authenticates the user through the Movere region closest to the geographical location, (e.g. West Europe or Australia East).
- Movere does this by replicating the user logon information, (the information provided to Movere when registering a user account), across all Movere regions.
- After the user is authenticated and emulates a tenant in the Movere portal (located in West US 2), the user is routed from the region that authenticated the logon, to the region in which the user data resides. This is required, because all scan data is only accessible from the region in which the database resides.
- Movere retires data that's greater than 31 days old. When the Days Since Last Activity setting exceeds 31 days for a device, Movere flags the device as retired, and moves the machine from all Azure sizing views. Note that the data isn't lost. To update from retired to active, you rescan a device. Periodic scanning ensures that the last activity date is continually updated.
- Data stored in the Movere cloud is retained in a locked state for up to 15 days after expiration of a Customer Tenant. During this time, no user can access data, including customers, partner users, and Movere support. Customer data, and backed-up customer data, is deleted from Movere on the 16th day.
Password storage
Movere doesn't store any passwords in the bots.
- Bots are distributed without any passwords hashes.
- The Console only sends passwords securely in memory to the bots, when secondary databases (such as SQL Server), are detected on target devices, and the NT Authority account on the target cannot access the secondary database.
- The Console only sends passwords to the bots if the target devices have active and valid token files. Tokens are needed for secure communications.
- The token file is used for uploading payloads to the cloud.
- The token2 file is used for transferring payloads from a target device to the Console.
- At no point during the scanning process will password hashes be stored in the bots, or on the target machines.
- Similarly, Movere prohibits the propagation of credentials with Domain Admin privileges to the bots for security purposes, when running as a service. This means that Domain Admin credentials can be used to run the Console and scan Windows devices, but they're not sent to the bots to collect secondary data.
Data security
Movere constantly evolves to include new features and security practices aimed at protecting against malicious attacks. Movere adheres to Cloud Native Computing Foundation (CNCF) principles. Data is secured as summarized in the table.
| Data | Secured with |
|---|---|
| Passwords | ASP.NET encryption (see PBKDF2) |
| Tool credential encryption | SHA 256 |
| Database/server | Azure SOC reports |
| Data at rest (data collected in the customer environment) | PGP RSA 2048 encryption |
| Data in transit | PGP + TLS 1.2 |
| Data in browser | Qlik proxies, which use TLS 1.2 |
Note that:
- Standard cryptographic APIs are used for cryptographic processing, and industry standard encryption algorithms are used.
- Encryption keys meet security requirements: Symmetric keys are at least 256-bits. Asymmetric public/private key pairs are 2048-bits.
- The application provides a mechanism to detect and lock unauthorized access to cryptographic keys.
- All access from the application to the database is restricted, and allows for only the least amount of functionality required to accomplish tasks.
- The application provides a mechanism to disable user accounts after three sign in failures.
- The application can log all user access to confidential information.
Data-in-transit
- For encryption, the Movere Console includes a PGP key that is unique to each customer.
- The key is 2048-bit long, and is used to encrypt (in memory) all data collected prior to its transmission, either back to the Movere Console for uploading to the cloud, or directly from the target device to the cloud.
- It's referred to as the public key, and can only be used to encrypt data.
- To decrypt the data, the encrypted payload needs to be uploaded to the cloud where specialized APIs identify the user, match the user account to a customer tenant, and retrieve the corresponding private PGP key from a repository managed by Movere.
- The user account uploading data using the Movere Console needs Write access. After the user account authenticates, the user is issued a long-lived token (token.txt). The token is valid for 12 hours, and is used for every upload via the Movere console.
Data-at-rest (in Azure)
After customer data reaches Movere cloud servers, it's decrypted and stored in memory, before being saved to a series of databases (depending of the data type).-
- Wherever available, data stored in databases such as SQL Azure and Azure Data Warehouse is encrypted, using native technologies such as Transparent Data Encryption (TDE).
- For data stored in document databases, data is encrypted, both at rest and in transit, using native technologies in Azure.
- All connections between APIs and data stores are encrypted and secured. The only exceptions are data stores, where either the data that is stored is ephemeral (cached or used for live processing), or doesn't contain user data.
Data-for-presentation
To render data via the browser, Movere uses Qlik Sense, a BI tool that enables users to navigate data using visualizations, and to leverage large volumes of data without compromising load time. Qlik uses proprietary in-memory technology, so no data is saved on disk while being visualized in the browser. In addition, the connection between Movere’s web servers and the user browser is encrypted using TLS 1.2.
Next steps
Start scanning Windows devices.