Side Loads

If you have records already loaded in your ILS, you should be able to export the existing files and side load them into Aspen. However, your eContent vendor should be able to provide you with a new up-to-date file containing all records in their collection. Reach out to the vendor's support for more information. Sometimes, searching "vendor + MARC records" on their website or from a search engine can also help get you in the right place.

The records must be in .mrc, .marc, or .mrk format.

Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please unzip the files first before uploading.

Note: Records must have an 001 field or other field with unique record identifiers.

Aspen will look at the 856 and only load links where first indicator is 4, 0, or blank and the second indicator is 0, 1, 4, or blank.

To turn on the side loads module, go to Aspen Administration > System Administration > Modules > Side Loads > Click Enabled and Save

Go to Aspen Administration > System Administration > Permissions. 

The permission that corresponds to side loads is in the Cataloging & Econtent drop down and is: Administer Side Loads - Controls if the user can administer side loads.

Part 1: Create New Side Load Settings

First, navigate to the Side Loads section of Aspen Administration. Click Settings, then click Add New.

1. Give the side load a name. This will be the display name for the collection in your catalog. Avoid using special characters in the name such as :, &, (, etc.

Example: Kanopy

2. Decide how the access button for the records should display. 

The choices for display are:

1. Define text in the Access Button Label field. The default text is "Access Online," but can be modified in this field. (Example: "Check out on Kanopy")
2. Alternatively, you may enable the checkbox for Use Link Text For Button Label. This will pull text from the 856 field.

As a best practice, keep the amount of text in your access button to a minimum.

If you've opted to pull text from the 856 field for the access button labels and you wish to modify the label text, it's recommended that you modify the records themselves rather than use Aspen's translation tool.

3. In Record URL Component, type in the name of your side load with no spaces or special characters.

Example: LearningExpress

The record URL component becomes a part of the access URL for these records. That might look something like:

library.aspendiscovery.org/LearningExpress/194758

4. Update the MARC Path field. Leave the text in this field as-is with the exception of {sideload_name}. Delete this portion of the text and replace it with your sideload name with no spaces or special characters. Make sure the curly brackets are removed as well.

Example: /data/aspen-discovery/library.production/LearningExpress/marc

This creates the directories where the side load files will be stored on the server.

5. Save your settings.

Below is an example of a Kanopy side load setting:

Optional Settings

The following settings are not necessary to change in most cases, but may be of interest depending on your needs/preferences.

Show Status: Will show availability if checked; this is checked on by default.

Filenames to Include: Leave as-is unless you plan to upload an .mrk file rather than an .mrc or .marc file. If this is the case, copy and paste the following text into this field:    .*\.(ma?rc|mrk)

Format Information: To have the side loaded records display as a specific format, change "Load Format From" to "Specified Value," then enter in the specified format, select the format category, and apply any format boost if desired.

Example:

Side Load Settings for Physical Collections

As of the 24.06 Aspen release, libraries can use the side load settings for physical collections as well. This is great for special collections such as MediaBank, Kits, Bookable collections, etc.

For physical collections, follow the same setup instructions above, but make sure to uncheck the Convert Format to eContent setting. Aspen will then look at the 856 field and only load links with the following criteria:

First indicator is 4, 0, or blank and the second indicator is 0, 1, 4, or blank.

Part 2: Scope Your Side Load

After saving the side load setting, Aspen will create a corresponding Scope setting. Scope settings tell Aspen which libraries should have access to these records.

1. From Aspen Administration, navigate to Side Loads > Scopes, then click on the corresponding scope to edit. 
2. Toward the bottom of the settings, click Add New to set all the applicable libraries that should have access to this side load. To speed up this process, click Apply to All Libraries and/or Apply to All Locations and either leave as-is or delete any libraries/locations that should not have access.
   
   Alternatively, in Primary Configuration > Locations, you can edit the Location settings to automatically refer to the sideloaded content for the parent library. If your locations are set up this way, selecting Locations in the side load scope settings won't be necessary; simply select the appropriate Library.

3. Save your changes.

Part 3: Adding Files

The setting has been created and the scope has been set. Now, you can add files!

1. Navigate to Side Loads > Settings > edit the setting.

2. Click Upload MARC file.

3. Click Select MARC File to choose and upload the file containing your records. The accepted file formats are .mrc and .marc. The .mrk format will also work, but requires adjustment of the Filenames to Incude field within the side load settings. Zipped files cannot be uploaded from the Aspen interface at this time. If you have a .zip file containing numerous MARC file collections, please reach out to your support company for assistance, as they may be able to load the files for you on the server. 

Important: only check the box for "Replace Existing Files?" if you want to delete ALL previously uploaded files and replace them with your current file. Otherwise, leave unchecked for new files and additional files.

4. Click Upload File. Repeat this process for any additional files as needed.

5. Once files are uploaded, they will go through an initial indexing process. To check the status of indexing, navigate to the Indexing Log within the Side Loads menu in Aspen administration. The records will show up in the "Products Added" column. If the records still don't appear in search results after being indexed, return to the side load settings and check the box for "Run Full Update," then save. 

Checking this box forces Aspen to index the records again. This time, the records will show up in the "Products Updated" column, and the items will now show in the catalog search results.

Add A New Scope

In Aspen, "scope" refers to "where the settings should be applied." In some cases, you may need to create multiple scopes for the same side load settings. Some common scenarios include:

• You have one set of Kanopy records, but each library will need their own unique access URLs applied. Rather than uploading what is essentially the same collection multiple times, upload the files once and then create multiple scopes with URL matching and replacement.
• You have a regular catalog and a children's catalog. For the children's catalog, a scope can be used to "include children's materials only."

To add a new scope:

1. Head to Side Loads > Scopes > click Add New.

2. Select the appropriate side load setting to associate with this scope.

3. Enter a name for the new scope.

Examples:

• If your setup includes multiple libraries and the URLs of the side load will be different for each library, you may want to name your scopes according to the location names -- for example, Main Branch, East Branch, and so on -- one for each separate catalog. 
• If you have a separate kid's catalog, you may want to have a scope for "All Records" and another for "Children's Records Only"

4. Adjust/enter in any settings as needed.

5. Click Save Changes and Stay Here.

6. Before leaving, select the applicable Libraries and/or Locations. Click Add New to add the libraries/locations individually. To speed up this process, click Apply to All Libraries and/or Apply to All Locations to automatically apply to all libraries/locations. Libraries/Locations can be deleted individually if needed.

Alternatively, in Primary Configuration > Locations, you can edit the Location settings to automatically refer to the sideloaded content for the parent library. If your locations are set up this way, you can simply select the Library in the side load scope settings and disregard the Locations.

7. Save your changes.

To verify that the settings are correct for each catalog, you may navigate to Aspen Admin > Primary Configuration > Library Systems, then click into the library system setting. Scroll toward the bottom to Side Loaded eContent Scopes where you should see all side loads assigned to this library system.

Scope by Audience

As of the 24.04 release, Aspen has the ability to scope side loads by age/audience.

To view your options, go to Aspen Administration > Side Loads > Scopes. Click into your existing scope setting or click Add New.

Kid's/Youth catalogs are a common use case for scoping side loaded content by audience. This way, libraries have the option to show off their full collection in the main catalog and only include juvenile or teen materials in the youth catalog.

The available options are:

• Include Adult Titles
• Include Teen Titles
• Include Kids Titles

These options follow Aspen's bib-level rules for determining audience the 006/008 fields. Click here to read more about these rules. 

Run a Full Reindex

To reload the side load records with your new scope settings applied:

1. Return to Side Loads > Settings. Click into the applicable setting.

2. Scroll down and check the box for Run Full Update.

3. Click Save.

You can see the indexing progress in Side Loads > Indexing Log.

Note: If changes to scoping aren't reflected in search results after the indexing is complete, click into Side Loads > Settings > check Run Full Update a second time. Depending on the timing of when settings are modified and when the indexer runs, it can take two full updates for the records to show changes. 

Depending on the type of URL in the side load, Aspen has multiple different options for changing the URL. 

You might need to change the URL in the MARC records if:

• Multiple libraries use the same set of MARC files, but each library catalog needs to have a different URL to authorize the records
• You have a generic set of MARC records where you need to add your library's URL instead 

In the side load's scopes, you have a number of options to change the URL. 

Please note that the following examples are not exhaustive and may not work for every set of records. We're unable to provide documentation for every possible side load resource. If you need assistance with URL modification, please reach out to your support company.

Switch the URL with URL to Match and URL Replacement

Example:

In this example, I have three scoped catalogs sharing the same set of Kanopy MARC records. The URL in the MARC records is: https://mainlibrary.kanopy.com

First, I added three individual scopes: "All Records - Main," "All Records - East," and "All Records - West"

For Main, I can leave it as-is because the URLs in the MARC records are already set to go to Main's Kanopy records.

For East and West, I will need to tell Aspen to change the URLs in the MARC records to their respective library's URLs. 

You can do this with either URL components or the full URLs to change. 

In the East example, I told Aspen anywhere in the URL that is says 'mainlibrary', change this to 'eastlibrary'

In the West example, I told Aspen anywhere in the MARC records it sees https://mainlibrary.kanopy.com to change it to https://westlibrary.kanopy.com

Either of these will work. 

Replace URL with EZProxy or Other Authentication Link

To replace an entire URL in the MARC records to an EZProxy or other authentication link, you can replace the URL completely by using regex.

In this example, the URL to Match is set to (.*) to change the entire URL in the MARC records to the URL Replacement that is added which is an EZProxy link.

Replace a Part of a Generic URL

If the URL in your MARC records is generic, you can target certain aspects of the URL.

Example: 

In this example, the URL is https://link.gale.com/apps/PTCP?u=[LOCATIONID]&hub=teacher

You would need to tell Aspen to fill in your location's ID to authenticate this URL.

In this example, URL To Match is (\[LOCATIONID\]) and the URL replacement is the library's location ID. 

Add a Beginning to All URLs 

Example: 

You want to add a beginning to all of the URLs, like http://proxy.com/login?url=

In this example, in the scope, in the URL to Match (Regular Expression) field, put in (.*)

Then, in the URL Replacement (Regular Expression) field, put in the beginning URL with $1 at the end, example: http://proxy.com/login?url=$1

When making URL scoping changes, you will have to run 1-2 full reindexes before the changes apply in your catalog

Depending on your library's workflow and how often the vendor updates or adds new records, you may need to occasionally add new records to your side loads -- perhaps monthly or quarterly. 

To add a new file to your side load:

1. Head to Aspen Administration > Side Loads > Settings > Click into the Side Load > click Upload MARC File. 

To see which files have already been loaded, click View MARC files. Before you make any changes, you may want to view and download the current files as a backup.

2. Click Select MARC File to choose the file containing your records.

IMPORTANT: Clicking "Replace Existing Files?" will delete all existing records and replace them with the new file. DO NOT click "Replace Existing Files?" if you are just uploading new titles and the new file does not include the current records.

3. Click Upload File.

4. Return to Side Loads > Settings > edit the side load setting > scroll down and check Run Full Update. 

5. Double check that your new records are loaded in the catalog. You can check on the side load record indexing progress from Side Loads > Indexing Log.

If the indexing has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

By default, Aspen will use the bib-level format determination rules to determine format for side loads. This configuration is best if the records within a side load contain a variety of formats (eBook, eAudio, etc). Click here to read more about how Aspen determines format for side loads.

To instead set a specific format for all records in the side load, here's how:

1. Go to Side Loads > Settings > Edit the side load setting you want to modify.
2. Scroll to the Format Information section.
3. Under Load Format From, switch this from Bib Record to Specified Value. (Note that the "Item Record" option is not currently a valid option)
4. If your marc records are for physical records, uncheck Convert Format to eContent
5. In Specified Format, enter the format label you want to appear for all records in the side load. In the below example, a library has side loaded records for their museum passes and wants the format button to reflect what they are.

5. Select a format category and/or format boost.
6. Check the Run Full Update box below this section, then save your changes. After the side loaded records reindex, you should see the format labels updated. You can check on the side load reindexing progress under Side Loads > Indexing Log.

Remove Specific Records

To remove specific records from a side load, go to Side Loads > Settings > click into the side load

In Deleted Records, you can add the record IDs from the Record Number Tag, by default the 001. You can add these one on each line and/or separated by commas. 

Once added, Aspen will ignore these record IDs in the side load MARC file. 

This is easy to manage within Aspen but can become a bit cumbersome as you continue to add more and more records to delete. 

Replace Existing Files

Some libraries prefer to work with one MARC file at a time and upload a new file for the entire collection each time changes are made. 

1. To access your current file, you can click on Side Loads > Settings > click into the side load > View MARC files

2. Click on the hyperlinked file name to download the file.

3. In an editing tool, like MARC Edit, you can remove the records you want to delete. 

4. Once you make all your deletions and save the new file, return to the side load settings and click Upload MARC file. 

5. Select your new MARC file.

6. Check the box for "Replace Exisiting Files?"

Important: This will delete your current side load records and replace them with your newly uploaded file.

7. Click Upload File.

8. Once uploaded, go back to Side Loads > Settings > Scroll down and check Run Full Update, then Save.

Double check that your new records are loaded in the catalog.  You can check your the indexing progress of your records in Side Loads > Indexing Log.

If the indexer has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

If you go into the Side Load and click Delete without first clearing out the records and/or the scopes, you will most likely see an error and may need to contact support.

If you are looking to remove a side load completely, we recommend doing this one of two ways:

Remove the Scope First

First, go to Primary Configuration > Library Systems > click into the library system you'd like to remove it from > scroll to the bottom toward Side Load eContent Scopes > find the Side Load Scope and click Delete.

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete

Remove the MARC Files(s) First (best for removing the same side load on multiple library catalogs)

First, go to Side Loads > Settings > click into the side load you want to remove > View MARC Files

Here you will see a list of all the MARC File(s) associated with that side load. Click Delete on each file.

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete

When deciding how to manage side loads in a consortium, there are a few factors to consider.

Permissions/Access to Settings

The way that permissions are set up, there is no current way to scope side load settings by library where the library can only see their own settings. Because of this, a library consortium may consider:

• Give the permission out to libraries after training has taken place and clearly labeling side loads by library name.
• Form a small group of cataloging or other staff to manage the side loads for the entire consortium. The group could survey the libraries to see what resources they would like to side load, receive the files from the libraries and manage updates. These staff members would be given access to all side loads.
• Manage all the side loads at the consortial-level. The consortium staff could survey the libraries to see what resources they would like to side load, receive the files from the libraries and manage updates. These staff members would be given access to all side loads.

Sharing a Side Load Setting with the Same Files

If multiple libraries subscibe to the same vendor and all their holdings are the same, then you can set up one setting for those libraries to share and use one set of files. This saves time instead of having to set up several individual settings. 

If this is the case and you want to update the URLs to all go to their own individual library path, then you can apply various scopes to the one setting.

Example:

In this example, I have three scoped catalogs sharing the same set of Kanopy MARC records. The URL in the MARC records is: https://mainlibrary.kanopy.com

If you are unsure what the default URL is, check the 856 of the MARC records.

First, I added three individual scopes: "All Records - Main," "All Records - East," and "All Records - West"

For Main, I can leave it as-is because the URLs in the MARC records are already set to go to Main's Kanopy records.

For East and West, I will need to tell Aspen to change the URLs in the MARC records to their respective library's URLs. 

You can do this with either URL components or the full URLs to change. 

In the East example, I told Aspen anywhere in the URL that is says 'mainlibrary', change this to 'eastlibrary'

In the West example, I told Aspen anywhere in the MARC records it sees https://mainlibrary.kanopy.com to change it to https://westlibrary.kanopy.com

Either of these will work. 

Setting Up Unique Record URL Components and MARC Paths for Each Sideload

If you do end up having the same vendor (i.e. Gale) and the library's subscribe to different packages, therefore having different record sets, then a seperate setting and scope need to be set up for each record set.

When doing this, you must have unique naming conventions in the MARC Path and the the Record URL Component.

It's common to use the library name in the settings.

Example:

Side Load Setting 1:

• Name: Gale - Main
• Record URL Component: maingale
• MARC Path: /data/aspen-discovery/library.production/maingale/marc

Side Load Setting 2:

• Name: Gale - East
• Record URL Component: eastgale
• MARC Path: /data/aspen-discovery/library.production/eastgale/marc

Side Load Setting 3:

• Name: Gale - West
• Record URL Component: westgale
• MARC Path: /data/aspen-discovery/library.production/westgale/marc

To see how many users are accessing side loaded records via Aspen, you can check Side Loads > Dashboard.

In general, if a vendor can provide you with MARC records, they can be side loaded into Aspen. 

Here are some commonly side loaded resources:

• BookFlix Books and Movies
• Brainfuse
• Chilton's
• Comics Plus
• Creativebug
• Credo Reference
• Gale Courses, Gale eBooks
• Flipster
• Freading
• Infobase
• Kanopy
• LinkedIn Learning
• LOTE4Kids
• O'Reilly
• Peterson's Test and Prep Center
• PressReader
• Project Gutenberg
• UnglueIT
• Worldbook

However, the side load function has also been used to load in records for other resources, such as museum or park passes.

Aspen can be set to ignore a number of econtent providers, in favor of using the direct API connection and/or sideloading directly into Aspen. 

To set which eContent records are automatically suppressed from your ILS, go to Aspen Administration > ILS Integration > Indexing Profiles > Suppress Records With Urls Matching.

This means that any records containing URLs matching these domains in the 856u will be automatically suppressed in the catalog.

You can add additional URLs here separated with pipes. 

Once econtent side loads are set up successfully, they will appear in the eContent Collection facet along with any other side loads and any econtent coming in via API. 

Physical side loads will show up in the Available Now At facet.

If you don't see the eContent Collection facet, check your facet set up in Aspen Administration > Catalog/Grouped Works > Group Work Facets. 

You'll want to make sure the "eContent Collection" facet is added and "Show on Results Page" should be checked.

If this facet is still not showing, you may want to check that your side load is scoped correctly. Head to Aspen Administration > Side Loads > Scopes, edit the applicable scope setting, and make sure that all desired libraries and locations are checked.

You can also check within the library systems to verify the scoping has been set up correctly. Aspen Administration > Primary Configuration > Library Systems > click into the library you want to check > scroll down to Side Loaded eContent Scopes. This will show all side loaded content applied to that libray system. 

If your side loaded records aren't showing up in search results, try the following steps.

1. Verify that your record file(s) uploaded successfully.

Edit your side load settings and click View MARC Files. If your file was successfully added, you will see the file name along with the date added and file size.

If you see a file here but there is no valid extension shown in the file name (.mrc, .marc, etc), double check that your file is in a valid file format and re-upload the file. Sometimes manually adding the valid extension to the file name on your device before uploading will fix the issue.

Note: Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please try unzipping the files first.

2. Verify the records have record numbers.

By default, Aspen is looking for a unique record number in the 001 field. If no record numbers are present, unique record numbers will need to be added to the records. If record numbers are present in a field other than the 001, you can adjust where Aspen looks for the record number in the side load settings > Record Number Tag / Record Number Subfield.

3. Verify the records are encoded according to your settings.

By default, side loads are set up for UTF-8 encoding. If your MARC file uses another type of encoding, you can adjust this in the side load settings > MARC Encoding > select your file's encoding from the dropdown. Alternatively, you can open up your file in a program like MarcEdit and compile the records to UTF-8 encoding, then try re-uploading the file.

4. Force a Full Update.

If you verified that a file was successfully uploaded and it has the correct file extension, you may just need to force the records to index again. 

Edit the side load setting, scroll down, check the box for "Run Full Update," then save. The next time the side load indexing process runs, Aspen will reindex the records and they should appear in search results within a few minutes (depending on how big the collection is).

5. Check the Indexing Log.

If you have done all of the above and your records are still not showing, go to Side Loads > Indexing Log and check to see if your side load log has any errors preventing the files from successfully indexing.

If you have just forced a full update, sometimes the process may simply still be running. Large collections can sometimes take up to 15 minutes to finish a full index.

When setting up sideloads, you might come across an error like this in your Side Loads > Indexing Log and notice your files aren't loading:

ERROR: Marc Directory /data/aspen-discovery/library.production/sideloadname/marc did not exist

This error is usually caused by creating a new side load and changing the information in the {sideload_name} in the MARC Path or the Individual MARC Path (example: you changed the {sideload_name} to gale but then you decide you want to change it to galeebooks instead). These fields are telling Aspen where to store this information on the server. If Aspen starts this process and then the side load name is changed again, Aspen will usually display that Marc Directory error. 

You can fix this in one of two different ways:

• First, you could delete that side load setting and start over by creating a new side load with a new, never before used {sideload_name}.
• Second, you could put in a support ticket with your Aspen team to adjust the path on the server.