This topic is intended for administrators and developers with administration access rights in Optimizely.
While Optimizely Search & Navigation pushes internal content to the search engine instantly, a connector crawls external websites on a specified schedule.
To access the Connectors screen, from the top menu Group of options that appears in the uppermost part of the user interface when logged in. The global menu displays products and systems integrated with your website. It also displays links to the user guide and user settings, and to a global search. See the User interface topic for further information., select Search & Navigation > Configure > Connectors. To view on-screen help, click Show Help in the top right corner. The following points supplement the on-screen help.
- By default, two connector types are available: Crawler and RSS/Atom. The connector type determines which configuration options appear below the Type drop-down list.
See also: Media Types. Optimizely Search & Navigation excludes the following media types from indexing by default.
Click Advanced fine tuning of indexing to further limit indexing.
- Exclude query strings that are part of a link. For example, exclude crawling campaign tracking parameters (such as
utm_source, used by Google Campaigns) to avoid unintentionally updating a campaign counter.
- Specify parts of a website to crawl but not index, or to not crawl at all. You may want to crawl but not index index search links to other pages, but not the content on those pages.
- Specifying an indexing interval.
- Although you set a schedule in local time, it is converted to coordinated universal time (UTC) so it occurs at the same time regardless of server location. However, you must manually adjust local time when needed, such as for daylight savings time.
Viewing connectors and indexing jobs
The connectors list (at the bottom of the screen) shows status and scheduling information for all indexing jobs.
- You can manually refresh a connector's indexing status. If completed, last completion time appears.
- You can edit or delete any connector from its context menu. For example, you can update its schedule.
- You can manually start and stop indexing jobs.
Updating a crawler's start URL
If you edit a connector's start URL, the crawl removes from the index all items that were previously indexed under the old URL.