CDNs

The CDN functionality encompasses many changes. Broadly speaking it is the ability to store and serve "transitory" files (e.g. CSS, images, etc) from somewhere besides the application server. Previously, FarCry's support for this kind of thing has been limited to configuring where on the web server files are stored.

The following use cases should now be supported:

• using local file system, S3, and FTP storage interchangeably
• configuring the 4 broad "types" of FarCry files separately (cache, images, secure files, public files)
• migrating files from an existing location to a new location

These changes introduce the idea of a "location", which is a discrete configuration that is later referred to by name. FarCry core uses five locations (cache, images, privatefiles, publicfiles, archive), and you should consider their access characteristics when you're looking at moving to a CDN - you may decide that some of these locations shouldn't be on a CDN because of security, serving responsiveness or file access speed.

To change FarCry's default behaviour you would override one (or all) of these in /projects/yourproject/config/_serverSpecificVarsAfterInit.cfm.

This article covers the following aspects of implementing FarCry CDN features:

CDN Types

FarCry core includes three configurable types of CDN.

Local

Use cases:

• store files in project webroot and server them with the application web server (default)
• store files outside the project root and server with a web server directory alias
• store files on a shared network drive
• serve files from a separate domain

Configuration properties:

As an example, here is the default cache setup:

<cfset application.fc.lib.cdn.setLocation(
	name="cache",
	cdn="local",
	fullpath=application.fc.lib.cdn.normalizePath(application.path.cache),
	urlpath=application.fc.lib.cdn.normalizePath(application.url.cache)
) />

S3

Use cases:

• store and serve public files from S3
• store and serve secured files from S3 (using temporary URLs)

Note that we've found latency to be an issue for resources that need to be served quickly (i.e. CSS, JS, images). If you want to host such files on S3 you should use something like CloudFront to improve responsiveness.

Configuration properties:

Here is an example where I have overridden the location of FarCry images. Notice that first I create a new, non-standard location that has the default details. This is useful if you want to use the built in migration tool to move local files to a CDN.

<cfset application.fc.lib.cdn.setLocation(
    name="images_old",
    cdn="local",
    locationinfo=application.fc.lib.cdn.getLocation("images")
) />
<cfset application.fc.lib.cdn.setLocation(
    name="images",
    cdn="s3",
    accessKeyId="ABCDEFGHIJKLMNOPQRST",
    awsSecretKey="ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCD",
    bucket="your-cdn-test",
    region="ap-southeast-2",
    security="public",
    pathPrefix="/images",
    admins=["you@example.com.au"]
) />

FTP

Use cases:

• storing files on a separate FTP server
• commercial CDNs like Limelight, which provide FTP access

Here is an example of overriding the public files location to put files on an FTP accessible CDN (Limelight in this case). Once again, you can see that I have copied the default location to a new non-standard config so that I can use it in the migration tool later.

<cfset application.fc.lib.cdn.setLocation(
    name="publicfiles_old",
    cdn="local",
    locationinfo=application.fc.lib.cdn.getLocation("publicfiles")
) />
<cfset application.fc.lib.cdn.setLocation(
    name="publicfiles",
    cdn="ftp",
    server="example.upload.llnw.net",
    username="example-user",
    password="ABCDEFGHIJK",
    ftpPathPrefix="/content/cdn-test/publicfiles",
    urlPathPrefix="//example.xx.llnwd.net/v1/cdn-test/publicfiles",
    passive=true
) />

Migrating an Existing Application to a CDN

This is a staged process designed to keep the site available as much as possible.

Stage 1: Preparing the Code

Most CDNs serve content from a domain other than the application one (the "local" CDN being the obvious exception). In many cases FarCry will take care of using the relevant domain automatically, but there are two cases where you may be referring directly to a file in your own code: files and images. You will need to make the following changes to prepare to a CDN that isn't on the same domain as the application:

files

All files should be linked to via /download.cfm, where FarCry will take care of file security and location. The format for these links is:

/download.cfm?downloadfile=[content objectid]&typename=[content typename]&fieldname=[file field]

images

Images in code should either:

• be prefixed with the result of application.fapi.getImageWebRoot(); or
• use the path returned from application.fc.lib.cdn.ioGetFileLocation() (see below)

Stage 2: Migrating Existing Files

1. In /projects/yourproject/config/_serverSpecificVarsAfterInit.cfm add the new CDN. In Stage 3 you will be making it the default for core locations, but first you need to add it as a new location. This way you can copy existing files to it while still serving them from the old location. Refer to CDN Types above to see examples of this. Remember to name your new locations something non-standard (like new-images).
2. After updating your application, open the CDN Migration Tool (Webtop -> Admin -> Developer Utilities -> Diagnostics -> CDN Migration Tool). Select the old and new locations and click Compare. NOTE: you will need to enter filter by /images for the images location.
3. Select the files you want to upload and click Copy Files

Stage 3: Switching the Application to the CDN

Go back to /projects/yourproject/config/serverSpecificVarsAfterInit.cfm and update the CDN locations with the core names. Update the application.

Congratulations! Your application is now serving content from the CDN!

Stage 4: Cleaning Up Files Uploaded During Migration (optional)

This stage is only necessary if you couldn't lock down content during the migration. In this case you should now have the core locations using your CDN, and some non-standard locations that still refer to the old locations.

Now you can use the migration tool (from Stage 2) to copy missing files. The tool will highlight files that are already there and those that aren't.

Accessing CDNs in Your Own Code

There are situations where you will want to access the CDNs yourself for custom functionality. A couple of cases where we've done so:

• automatically generating archvies and uploading them to S3
• writing a migration utility to compare two CDN locations and copy selected files between them
• clearing out a CDN after doing tests on stage

This section describes the public API of the CDN library, which takes care of things like ensuring filename uniqueness, figuring out how to move files between different CDNs, and handling user uploads. Note that ALL public functions are prefixed with "io" to avoid conflicts with existing CFML functions.

You can access these function at application.fc.lib.cdn.

ioFileExists

Does what it says on the box. Checks a single location to see if a file exists.

<cfif application.fc.lib.cdn.ioFileExists(location="cache",file=sCacheFileName)>
    <cfreturn sCacheFileName />
</cfif>

ioFindFile

Searches the provided locations, and returns the first that contains the specified file, or an empty string if there isn't any.

<cfset currentLocation = application.fc.lib.cdn.ioFindFile(locations="privatefiles,publicfiles",file=arguments.stObject[arguments.stMetadata.name]) />

ioGetUniqueFilename

Returns a version of the specified filename which is unique among every listed location by appending numbers to the name. Note that it is rare to have to call this function directly, as all functions which put a file into a CDN have options to enforce filename uniqueness. However if you wish to change how FarCry enforces uniqueness, you can override this function in your project.

<cfset moveto = ioGetUniqueFilename(locations="privatefiles,publicfiles",file=newfile) />

ioGetFileSize

Returns the size of the file in bytes.

<cfoutput>Size: <span class="image-size">#round(application.fc.lib.cdn.ioGetFileSize(location="images",file=arguments.stMetadata.value)/1024)#</span>KB</cfoutput>

ioGetFileLocation

Returns serving information for the file. The result is a struct - either: method=redirect + path (URL) OR method=stream + path (local path).

<cfset stImage = application.fc.lib.cdn.ioGetFileLocation(location="images",file=arguments.stMetadata.value) />
<cfoutput><img src="#stImage.path#"></cfoutput>

ioWriteFile

Writes the specified data to a file.

<cfset stResult.filename = application.fc.lib.cdn.ioWriteFile(location="images",file=filename,data=newImage,datatype="image",quality=arguments.quality,nameconflict="makeunique",uniqueamong="images") />

ioReadFile

Reads from the specified file.

<cfimage action="info" source="#application.fc.lib.cdn.ioReadFile(location='images',file=stResult.value,datatype='image')#" structName="stImage" />

ioMoveFile

Moves the specified file between locations. Note that when moving a file between different CDN types, this function moves the file to the local temporary directory, then to the target location from there.

Note that while every argument is marked as optional, in practice you need:

• source_location and source_file OR source_localpath
• dest_location and dest_file OR dest_localpath

<cfset application.fc.lib.cdn.ioMoveFile(source_location="publicfiles",source_file=arguments.stObject[arguments.stMetadata.name],dest_location="privatefiles") />

ioCopyFile

Copies the specified file between locations. Note that when copying a file between different CDN types, this function copies the file to the local temporary directory, then moves it to the target location from there.

Note that while every argument is marked as optional, in practice you need:

• source_location and source_file OR source_localpath
• dest_location and dest_file OR dest_localpath

<cfset application.fc.lib.cdn.ioCopyFile(
    source_location="images",
    source_file=stLocal.stInstance.thumbnail,
    dest_location="archive",
    dest_file="/#stLocal.stInstance.typename#/#stLocal.stProps.archiveID#_thumb.#ListLast(stLocal.stInstance.thumbnail,'.')#"
) />

ioUploadFile

Uploads the a file to the specified location.

<cfset stResult.value = application.fc.lib.cdn.ioUploadFile(
    location="securefiles",
    destination=arguments.stMetadata.ftDestination,
    field="#stMetadata.FormFieldPrefix##stMetadata.Name#New",
    nameconflict="makeunique",
    uniqueamong="privatefiles,publicfiles",
    acceptedextensions=arguments.stMetadata.ftAllowedFileExtensions
) />

ioDeleteFile

Deletes the specified file.

<cfset application.fc.lib.cdn.ioDeleteFile(location="images",file="/#arguments.stObject[arguments.stMetadata.name]#") />

ioDirectoryExists

Checks that a specified directory exists. All functions which write to a CDN use this function to check whether the target directory needs to be created first.

<cfset application.fc.lib.cdn.ioDeleteFile(location="images",file="/#arguments.stObject[arguments.stMetadata.name]#") />

ioDirectoryExists

Checks that a specified directory exists. All CDN functions which create a file already perform internal checks to find out if a directory needs to be created first. As a result, there is no example in core of a call to this function.

<cfif application.fc.lib.cdn.ioDirectoryExists(location="images",file="/#stMetadata.ftDestination#")>
    <!--- something here --->
</cfif>

ioCreateDirectory

Creates the specified directory, including parent directories. All CDN functions which create a file already perform internal checks and create directories as necessary. As a result, there is no example in core of a call to this function.

<cfset application.fc.lib.cdn.ioCreateDirectory(location="images",file="/#stMetadata.ftDestination#") />

ioGetDirectoryListing

Returns a query containing the files under the specfied directory. The resulting query has a single field "file", which is the full path as would be passed into the other CDN functions. The results are recursive and do not include directories, except by implication.

It is worth noting that the "images" location is unusual in that it usually corresponds to the project WWW directory, and so a naive query to the "images" location will return everything in the webroot. In practice, listings of "images" should be filtered by /images.

<cfset qSourceFiles = application.fc.lib.cdn.ioGetDirectoryListing(location=form.source_location,dir=form.source_filter) />
<cfset qTargetFiles = application.fc.lib.cdn.ioGetDirectoryListing(location=form.target_location,dir=form.target_filter) />            

<cfset stFound = structnew() />
<cfloop query="qSourceFiles">
    <cfif not structkeyexists(stFound,qSourceFiles.file)>
        <cfset stFound[qSourceFiles.file] = 1 />
    <cfelse>
        <cfset stFound[qSourceFiles.file] = stFound[qSourceFiles.file] + 1 />
    </cfif>
</cfloop>
<cfloop query="qTargetFiles">
    <cfif not structkeyexists(stFound,qTargetFiles.file)>
        <cfset stFound[qTargetFiles.file] = 2 />
    <cfelse>
        <cfset stFound[qTargetFiles.file] = stFound[qTargetFiles.file] + 2 />
    </cfif>
</cfloop>

<cfloop collection="#stFound#" item="thisfile">
    <cfset queryaddrow(qFiles) />
    <cfset querysetcell(qFiles,"file",thisfile) />
    <cfset querysetcell(qFiles,"inSource",bitand(stFound[thisfile],1) eq 1) />
    <cfset querysetcell(qFiles,"inTarget",bitand(stFound[thisfile],2) eq 2) />
</cfloop>
            
<cfquery dbtype="query" name="qFiles">select * from qFiles order by file</cfquery>