Add export fsxn to cf program.

CloudFormation/Export-FSxN-CloudFormation/README.md

@@ -0,0 +1,64 @@
+# Export NetApp FSxN to a CloudFormation Template
+
+## Overview
+This folder provides a script that will create a CloudFormation template based on the current configuration of an existing FSx for ONTAP file system.
+
+## Prerequisites
+- An FSxN file system you want to create a CloudFormation template for.
+- An AWS account with permissions to "describe" the FSxN file system and its virtual storage machines, and volumes.
+- The AWS CLI installed and configured on your local machine. You can find instructions on how to do that [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html).
+
+## Running the script
+
+The script takes the following parameters:
+- `-f fs-id`: The ID of the FSxN file system you want to create the CloudFormation template for. This is a required parameter.
+- `-n name`: Is an optional name to be appended to all the volumes, svms and NetBIOS names. This is so you can test the CloudFormation template while the original machine is still running.
+
+The script will output the CloudFormation template in JSON format. You can redirect this output to a file if you want to save it.
+
+Note that since you can't retrieve credentials from the FSxN configuration the script will create
+parameters that will allow you to provide an AWS Secrets Manager secret that should contain the credentials.
+There will be one parameter for the password of the 'fsxadmin' account. That secret will just need one 'key'
+named "password" with the desired fsxadmin password. There will also be a parameter for each SVMs that has an
+Active Directory configured for it so you can provide a secret that should have a 'username' and 'password' key
+that will be used to join the SVM to the domain.
+
+An example run:
+```
+$ python export_fsxn_cf.py -f fs-0123456789abcdef0 -n test > fsxn_template.json
+Warning: Volume rvnw_vol_autogrow does not have a junction path yet it is required for a Cloudformation template so setting it to /rvnw_vol_autogrow
+Warning: Volume unixdata does not have a junction path yet it is required for a Cloudformation template so setting it to /unixdata
+Warning: Volume effictest2 is a DP volume and cannot have the StorageEfficiencyEnabled property, removing it from the CloudFormation template.
+Warning: Volume effictest2 is a DP volume and cannot have the SnapshotPolicy property, removing it from the CloudFormation template.
+Warning: Volume effictest2 is a DP volume and cannot have the SecurityStyle property, removing it from the CloudFormation template.
+Warning: Could not find root volume for SVM fsa. Setting the security style to UNIX
+```
+
+## Notes
+- For multi availability zone deployments, the script will do the following in regards to the Endpoint IP Address Range:
+    - If the file system is in the 198.19.0.0/16 address range (the AWS default), the script will not provide an address range forcing AWS to just allocate a new address range from the 198.19.0.0/16 CIDR block.
+    - If it isn't in the 198.19.0.0/16 address range then it will create a parameter so you can specify a new address range for testing purposes, with a default set to the current address range.
+- Since AWS requires you to provide a junction path when creating a volume, if the script finds a volume without a junction path it will set it to `/volume_name`. A warning message will be outputted if this happens to alert you.
+- Since AWS doesn't allow you to specify these parameters when creating a DP type volume, their current settings will be removed from the CloudFormation template:
+    - SecurityStyle
+    - SnapshotPolicy
+    - StorageEfficiencyEnabled
+- If, for some reason, the script can't find the attributes of the root volume of a SVM (unlikely but there are reasons how this can happen), it will set the security style of the SVM to 'NTFS' if the SVM has a Active Directory configuration, otherwise it will set it to a 'UNIX' security style. A warning message will be outputted if this happens to alert you.
+- Since AWS only allows an Active Directory Distinguished Name (DN) to start with "OU=", if the script finds a DN that doesn't start with "OU=" it will ignore it and will output a warning message to alert you. However the DN is set to "CN=Computers", which is the default DN that ONTAP will use when joining a domain to AD, it will not output an warning message with the assumption that by not providing a DN it will be set to "CN=Computers".
+- While some testing was performed, hence the `-n` option, not for all possible FSxN configurations were tested. If you run into any issues with the script, or have suggestions for improvements, please open an [issue](https://github.com/NetApp/FSx-ONTAP-samples-scripts/issues) on GitHub.
+
+## Author Information
+
+This repository is maintained by the contributors listed on [GitHub](https://github.com/NetApp/FSx-ONTAP-samples-scripts/graphs/contributors).
+
+## License
+
+Licensed under the Apache License, Version 2.0 (the "License").
+
+You may obtain a copy of the License at [apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an _"AS IS"_ basis, without WARRANTIES or conditions of any kind, either express or implied.
+
+See the License for the specific language governing permissions and limitations under the License.
+
+© 2025 NetApp, Inc. All Rights Reserved.

CloudFormation/Export-FSxN-CloudFormation/export_fsxn_cf.py

@@ -0,0 +1,192 @@
+#!/bin/python3
+#
+# This script takes an FSx for ONTAP file system ID as input and generates
+# a CloudFormation template for the file system, its volumes, and its
+# storage virtual machines. The output is printed to the console in JSON format.
+################################################################################
+
+import json
+import boto3
+import sys
+import optparse
+#
+# Get the file system ID from the command line
+parser = optparse.OptionParser()
+parser.add_option('-f', dest='filesystemId', help='The ID of the FSx for ONTAP file system to generate the CloudFormation template for.')
+parser.add_option('-n', dest='nameAppend', help='A string to append to the names of the resources in the CloudFormation template to make them unique. This is optional.')
+opts, args = parser.parse_args()
+
+if opts.filesystemId is None:
+    print("Error: -f option is required", file=sys.stderr)
+    sys.exit(1)
+filesystemId = opts.filesystemId
+nameAppend = opts.nameAppend if opts.nameAppend is not None else ""
+#
+# Create boto3 client for fsx and ec2.
+fsxClient = boto3.client('fsx')
+ec2Client = boto3.client('ec2')
+#
+# Get the file system details
+response = fsxClient.describe_file_systems(FileSystemIds=[filesystemId])
+if response.get('FileSystems') is None or len(response['FileSystems']) == 0:
+    print(f"No file system found with ID {filesystemId}", file=sys.stderr)
+    sys.exit(1)
+#
+# Build the CloudFormation template for the file system
+cfTemplate = {
+    "Description": f"FSx File System template for {filesystemId}.",
+    "Resources": {}
+}
+cfTemplate['Parameters'] = {
+    "fsxadminPassword": {
+        "Type": "String",
+        "Description": "The AWS Secrets Manager secret that has the password for the fsxadmin user. It should have a key named 'password' that contains the password."
+    }
+}
+fileSystem = response['FileSystems'][0]
+fsCfTemplate = {}
+fsCfTemplate['Type'] = 'AWS::FSx::FileSystem'
+fsCfTemplate['Properties'] = {}
+for prop in ['NetworkType', 'FileSystemType', 'KmsKeyId', 'StorageCapacity', 'SubnetIds', 'StorageType', 'Tags']:
+    if prop in fileSystem:
+        fsCfTemplate['Properties'][prop] = fileSystem[prop]
+#
+# Get the security groups from the ENIs.
+fsCfTemplate['Properties']['SecurityGroupIds'] = []
+securityGroups = {} # Use a dictionary to store the security groups to avoid duplicates.
+response = ec2Client.describe_network_interfaces(NetworkInterfaceIds=fileSystem['NetworkInterfaceIds'])
+for eni in response['NetworkInterfaces']:
+    for group in eni['Groups']:
+        securityGroups[group['GroupId']] = 1
+for group in securityGroups.keys():
+    fsCfTemplate['Properties']['SecurityGroupIds'].append(group)
+#
+# Copy the ONTAP configuration.
+fsCfTemplate['Properties']['OntapConfiguration'] = {"FsxAdminPassword": {"Fn::Sub": "{{resolve:secretsmanager:${fsxadminPassword}:SecretString:password}}"}}
+for prop in ['AutomaticBackupRetentionDays', 'DailyAutomaticBackupStartTime', 'DeploymentType',
+                 'EndpointIpAddressRange', 'EndpointIpv6AddressRange', 'PreferredSubnetId', 'RouteTableIds', 
+                 'WeeklyMaintenanceStartTime', 'HAPairs', 'ThroughputCapacityPerHAPair']:
+    if prop in fileSystem['OntapConfiguration']:
+        fsCfTemplate['Properties']['OntapConfiguration'][prop] = fileSystem['OntapConfiguration'][prop]
+if fileSystem['OntapConfiguration']['DiskIopsConfiguration']['Mode'] == 'AUTOMATIC':
+    fsCfTemplate['Properties']['OntapConfiguration']['DiskIopsConfiguration'] = {'Mode': 'AUTOMATIC'}
+else:
+    fsCfTemplate['Properties']['OntapConfiguration']['DiskIopsConfiguration'] = fileSystem['OntapConfiguration']['DiskIopsConfiguration']
+#
+# If using the default endpoint IP address range, remove it from the
+# CloudFormation template since AWS will automatically use a new default
+# address range if the 'EndpointIpAddressRange' is not specified.
+if 'EndpointIpAddressRange' in fsCfTemplate['Properties']['OntapConfiguration']:
+    if fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange'].startswith("198.19"):
+        del fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange']
+    else:
+        cfTemplate['Parameters']['endpointIpAddressRange'] = {
+             "Type": "String",
+             "Description": "The IP address range to use for the file system's endpoints.",
+             "Default": fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange']
+             }
+        fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange'] = {"Ref": "endpointIpAddressRange"}
+
+cfTemplate['Resources'].update({filesystemId.replace("-", ""): fsCfTemplate})
+#
+# Get all the volumes for the file system. Getting the volumes before the SVMs
+# since I need the list of volumes to get the security style of the root volume
+# for each SVMs.
+response = fsxClient.describe_volumes(Filters=[{'Name': 'file-system-id', 'Values': [filesystemId]}])
+volumes = response['Volumes']
+for volume in volumes:
+    if volume['OntapConfiguration']['StorageVirtualMachineRoot']:
+        continue
+    volumeCfTemplate = {}
+    volumeCfTemplate['Type'] = 'AWS::FSx::Volume'
+    volumeCfTemplate['Properties'] = {}
+    for property in ['Name', 'VolumeType', 'Tags']:
+        if property in volume:
+            volumeCfTemplate['Properties'][property] = volume[property]
+    volumeCfTemplate['Properties']['Name'] = volumeCfTemplate['Properties']['Name'] + nameAppend
+
+    volumeCfTemplate['Properties']['OntapConfiguration'] = {}
+    for property in ['AggregateName', 'CopyTagsToBackups', 'OntapVolumeType', 'SecurityStyle', 'SizeInMegabytes',
+                     'StorageEfficiencyEnabled', 'VolumeStyle', 'JunctionPath',
+                     'SnapshotPolicy', 'TieringPolicy', 'SnaplockConfiguration']:
+        if property in volume['OntapConfiguration']:
+            volumeCfTemplate['Properties']['OntapConfiguration'][property] = volume['OntapConfiguration'][property]
+    volumeCfTemplate['Properties']['OntapConfiguration']['StorageVirtualMachineId'] = {"Ref" : volume['OntapConfiguration']['StorageVirtualMachineId'].replace("-", "")}
+    #
+    # DP volumes can't have JunctionPath, StorageEfficiency, SnapshotPolicy or SecurityStyle properties
+    if volume['OntapConfiguration']['OntapVolumeType'] == 'DP':
+        for prop in ['JunctionPath', 'StorageEfficiencyEnabled', 'SnapshotPolicy', 'SecurityStyle']:
+            if prop in volumeCfTemplate['Properties']['OntapConfiguration']:
+                print(f"Warning: Volume {volume['Name']} is a DP volume and cannot have the {prop} property, removing it from the CloudFormation template.", file=sys.stderr)
+                del volumeCfTemplate['Properties']['OntapConfiguration'][prop]
+    else:
+        if 'JunctionPath' not in volumeCfTemplate['Properties']['OntapConfiguration']:
+            print(f"Warning: Volume {volume['Name']} does not have a junction path yet it is required for a CloudFormation template so setting it to /{volumeCfTemplate['Properties']['Name']}", file=sys.stderr)
+            volumeCfTemplate['Properties']['OntapConfiguration']['JunctionPath'] = "/" + volumeCfTemplate['Properties']['Name']
+
+    cfTemplate['Resources'].update({volume['VolumeId'].replace("-", ""): volumeCfTemplate})
+#
+# Get all the storage virtual machines for the file system.
+response = fsxClient.describe_storage_virtual_machines(Filters=[{'Name': 'file-system-id', 'Values': [filesystemId]}])
+for svm in response['StorageVirtualMachines']:
+    svmCfTemplate = {}
+    svmCfTemplate['Type'] = 'AWS::FSx::StorageVirtualMachine'
+    svmCfTemplate['Properties'] = {"FileSystemId": {"Ref" : filesystemId.replace("-", "")}}
+    for prop in ['ActiveDirectoryConfiguration', 'Name', 'RootVolumeSecurityStyle', 'Tags']:
+        if prop in svm:
+            svmCfTemplate['Properties'][prop] = svm[prop]
+    svmCfTemplate['Properties']['Name'] = svmCfTemplate['Properties']['Name'] + nameAppend
+
+    if 'ActiveDirectoryConfiguration' in svm:
+        if len(svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName']) > 10 and len(nameAppend) > 0:
+            svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'][:10] + nameAppend.upper()
+        else:
+            svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] + nameAppend.upper()
+        svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'][:15]
+        if 'SelfManagedActiveDirectoryConfiguration' in svm['ActiveDirectoryConfiguration']:
+            if 'OrganizationalUnitDistinguishedName' in svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']:
+                #
+                # Since CF can only handle organizational unit distinguish names that have a
+                # parent of OU, we need to check if the parent of the organizational unit is
+                # OU and if not, we need to remove the organizational unit distinguish name (DN)
+                # from the CloudFormation template and print a warning message.
+                dnParent=svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName'].split(",")[0]
+                dnParent = dnParent.split("=")[0]
+                if dnParent != "OU":
+                    #
+                    # The default value from ONTAP is 'CN=Computers' which does not have a
+                    # parent of OU, but CF requires that the parent is OU, therefore we will
+                    # just ignore the organizational unit distinguish name.
+                    if svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName'] != "CN=Computers":
+                        print(f'Warning: The organizational unit distinguish name for the SVM {svm["Name"]} is "{svm["ActiveDirectoryConfiguration"]["SelfManagedActiveDirectoryConfiguration"]["OrganizationalUnitDistinguishedName"]}" which does not have a parent of a OU and CloudFormation requires that, therefore the distinguished name is ignored. This will cause the SVM to be put into the "default" computer location', file=sys.stderr)
+                    del svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName'] 
+
+            secretParameterId = f'{svm["Name"].replace("-", "").replace("_", "")}AdminCredentials'
+            cfTemplate['Parameters'][secretParameterId] = {
+                "Type": "String",
+                "Description": f"The AWS Secrets Manager secret that has the Active Directory credentials for the {svm['Name']} storage virtual machine. It should have two keys named 'username' and 'password'."
+            }
+            svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['UserName'] = {"Fn::Sub": "{{resolve:secretsmanager:${" + secretParameterId + "}:SecretString:username}}"}
+            svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['Password'] = {"Fn::Sub": "{{resolve:secretsmanager:${" + secretParameterId + "}:SecretString:password}}"}
+    #
+    # Get the security style for the SVM's root volume. Assume the root volume is <svm_name>_root
+    for volume in volumes:
+        if volume['OntapConfiguration']['StorageVirtualMachineId'] == svm['StorageVirtualMachineId'] and volume['OntapConfiguration']['StorageVirtualMachineRoot']:
+            svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = volume['OntapConfiguration']['SecurityStyle']
+            break
+    if svmCfTemplate['Properties'].get('RootVolumeSecurityStyle') is None:
+        if 'ActiveDirectoryConfiguration' in svmCfTemplate['Properties']:
+            svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = 'NTFS'
+        else:
+            svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = 'UNIX'
+        print(f"Warning: Could not find root volume for SVM {svm['Name']}. Setting the security style to {svmCfTemplate['Properties']['RootVolumeSecurityStyle']}.", file=sys.stderr)
+    cfTemplate['Resources'].update({svm['StorageVirtualMachineId'].replace("-", ""): svmCfTemplate})
+
+cfTemplate['Outputs'] = {
+    "FileSystemId": {
+        "Description": "The ID of the FSx for ONTAP file system.",
+        "Value": {"Ref": filesystemId.replace("-", "")}
+    }
+}
+# Print the CloudFormation template in JSON format
+print(json.dumps(cfTemplate, indent=4))