Maximize performance by uploading your external data to Optimizely Graph

Optimizely Graph, a crucial component of the Optimizely Digital Experience Platform, excels at managing and interpreting complex data relationships efficiently. Its versatility in handling a diverse range of data types makes it a valuable asset for businesses of all sizes.

One of Optimizely Graph's standout features is its direct querying capability. Unlike traditional data management systems that involve multiple steps for data retrieval, Graph allows for direct access to custom data, significantly improving performance. This approach not only expedites data retrieval but also minimizes server load, leading to faster response times and more efficient data management.

Graph enhances the way businesses interact with their data, offering a streamlined approach that melds efficiency with performance. By integrating external data into this system, companies unlock the potential for a more dynamic and high-performance digital experience. This sets the stage for the technical process of uploading external data into Graph, a key step in leveraging data for strategic decisions and optimization in the digital space.

Content Preperation Essentials

Before you can synchronise data with Graph, the data needs to be prepared in a specific format. This preparation involves using the json format, which is essential for Optimizely Graph to ingest the data correctly.

We need to define a new content structure in Graph which supports the external data you wish to upload. You may mirror you external content 1:1 with Graph. Alternatvely its a good opportunity to normalize and massage your data into something more optimal. You may have property which stores a numeric ID representing a status. This may be useful for your external system but for consumers of the data, you may wish to serve a string representation instead which would be more meaningful and relevant. 

Let's consider the Thoughts feed from the Hidden Foundry website as an example. The properties we're interested in align with typical article metadata, such as Name, Description, Date, Author, and so on. This means we need to create an Article type in Graph along with the common properties.

{
  "feed": {
    "title": "Hidden Foundry Thoughts",
    "description": "Technical and Strategy thoughts from Hidden Foundry",
    "language": "en",
    "lastBuildDate": "Tue, 28 Nov 2023 11:03:13 Z",
    "link": "https://www.hiddenfoundry.com/thoughts/",
    "articles": [
      {
        "title": "Optimizely Graph vs Search & Navigation",
        "link": "https://www.hiddenfoundry.com/thoughts/optimizely-graph-vs-search-and-navigation/",
        "author": "[email protected]",
        "categories": ["Graph", "Upgrade"],
        "description": "Benefits and Limitations of Graph over Search & Navigation",
        "pubDate": "Fri, 03 Nov 2023 13:47:38 Z"
      },
      {
        "title": "Shifting Hurdles of CMS 12 Upgrade into Milestones",
        "link": "https://www.hiddenfoundry.com/thoughts/shifting-hurdles-of-cms-12-upgrade-into-milestones/",
        "author": "[email protected]",
        "categories": ["CMS 12", "Upgrade"],
        "description": "Critical parts of an upgrade can help you understand the different stages of an upgrade. Let's discuss.",
        "pubDate": "Tue, 31 Oct 2023 10:48:50 Z"
      },
      {
        "title": "Optimizely PaaS Core vs SaaS Core",
        "link": "https://www.hiddenfoundry.com/thoughts/optimizely-paas-core-vs-saas-core/",
        "author": "[email protected]",
        "categories": ["CMS", "Headless"],
        "description": "Optimizely Customers now have a choice between using CMS on either PaaS or SaaS",
        "pubDate": "Mon, 09 Oct 2023 23:13:26 Z"
      }
    ]
  }
}

Structuring Content

Graph supports the basic field types String, Int, Float, Boolean and [String], Optimizely goes one further and supports a 'Date' format. This needs to be in the ISO 8601 format.

In the example json below you will see how we create a new Content Type in Graph called Article and we have a selection of fields we want to create along with their type. This is the simpliest form of type creation, however you can also define multiple types in one json and also do linking between different types to support complex objects. It should be clear to see now Graph gives you the ability to create type of content you want.

To create these content types up in Graph, you'll need to use the Graph APIs.

{
    "propertyTypes": {
        "ProductLanguage": {
            "properties": {
                "DisplayName": {
                    "name": "DisplayName",
                    "type": "String"
                },
                "Name": {
                    "name": "Name",
                    "type": "String"
                }
            }
        }
    },
    "label": "feed",
    "languages": [
        "en"
    ],
    "contentTypes": {
        "Article": {
            "contentType": [],
            "properties": {
                "ContentType": {
                    "type": "[String]"
                },
                "title": {
                    "type": "String",
                    "searchable": true
                },
                "link": {
                    "type": "String"
                },
                "author": {
                    "type": "String",
                    "searchable": true
                },
                "description": {
                    "type": "String"
                },
                "pubDate": {
                    "type": "Date"
                },
                "categories": {
                    "type": "[String]",
                    "searchable": true
                }
            }
        }
    }
}

Content Upload Mechanics

Once the content types are defined, you can commence the data synchronisation process. It's important to ensure that the data is correctly prepared in the json format, which is a prerequisite for Optimizely Graph to properly ingest and process the data.  

To upload content requires your payload to be in NDJSON format (Newline Delimited JSON).

Aside from the fields we've created ourselves, you'll notice there are other fields tapped to the end of each content item which is required.

Status can either be 'Draft' or 'Published'.

RolesWithReadAccess is your typical Optimizely CMS roles register. In our case, we don't want any restriction on this content.

{ "index": {"_id": 1, "language_routing": "en" }}
{ "Id": "1", "title___searchable": "Optimizely Graph vs Search & Navigation", "link": "https://www.hiddenfoundry.com/thoughts/optimizely-graph-vs-search-and-navigation/", "author": "[email protected]", "categories": ["Graph", "Upgrade"], "description": "Benefits and Limitations of Graph over Search & Navigation", "pubDate": "2023-11-03T13:47:38Z", "Language": {"DisplayName": "English", "Name": "en"}, "ContentType": ["Article"], "Status": "Published", "RolesWithReadAccess": "Everyone" }

{ "index": {"_id": 2, "language_routing": "en" }}
{ "Id": "2", "title___searchable": "Shifting Hurdles of CMS 12 Upgrade into Milestones", "link": "https://www.hiddenfoundry.com/thoughts/shifting-hurdles-of-cms-12-upgrade-into-milestones/", "author": "[email protected]", "categories": ["CMS 12", "Upgrade"], "description": "Critical parts of an upgrade can help you understand the different stages of an upgrade. Let's discuss.", "pubDate": "2023-10-31T10:48:50Z", "Language": {"DisplayName": "English", "Name": "en"}, "ContentType": ["Article"], "Status": "Published", "RolesWithReadAccess": "Everyone" }

{ "index": {"_id": 3, "language_routing": "en" }}
{ "Id": "3", "title___searchable": "Optimizely PaaS Core vs SaaS Core", "link": "https://www.hiddenfoundry.com/thoughts/optimizely-paas-core-vs-saas-core/", "author": "[email protected]", "categories": ["CMS", "Headless"], "description": "Optimizely Customers now have a choice between using CMS on either PaaS or SaaS", "pubDate": "2023-10-09T23:13:26Z", "Language": {"DisplayName": "English", "Name": "en"}, "ContentType": ["Article"], "Status": "Published", "RolesWithReadAccess": "Everyone" }

Validating Upload and Querying

To verify that your external data has been successfully uploaded to Graph, execute a simple query using the Optimizely Graph UI. The query explorer can guide you through writing your initial query if you're new to this process.

Conclusion

We've guided you through the essential steps of integrating external data into Optimizely Graph, empowering you to wield the data insights that drive business success. This comprehensive guide has equipped you with the knowledge and tools to seamlessly connect external data sources, allowing you to analyze and optimize your business processes with unprecedented precision.

As you delve deeper into Optimizely Graph's capabilities, our team stands ready to support your journey, ensuring you maximize the potential of your data. Together, we can transform your organization into a data-driven enterprise, fueled by informed decision-making and innovation. Embrace the power of data, and watch your business flourish. Please do not hesitate to reach out to us as you embark on this optimizing journey. We're here to assist you every step of the way.