Deploy the Microsoft SharePoint Online connector

You can set up Google Cloud Search to return results from your organization's SharePoint Online content in addition to your Google Workspace content. You use the Google Cloud Search SharePoint Online connector and configure it to access a specific SharePoint Online data source.

Important considerations

Honored SharePoint settings

The Cloud Search SharePoint Online connector always honors the Search Visibility setting on SharePoint, which can't be overridden. For draft documents, the permissions on the user account (that the connector uses to access SharePoint Online) controls what draft documents are indexed and returned. If the account has only "Full Read" permissions, the connector honors the "Draft item visibility" settings on SharePoint.

You also configure the connector to limit results based on user account access. To apply security trimming for SharePoint content, you synchronize the following external identities with the Google Directory:

Office 365 / Azure AD Users
Office 365 / Azure AD security Groups
SharePoint groups (with Office 365 users and groups as members)

Search optimization

You can improve your users' experience by configuring the connector to return more relevant search results.

To use the API, set values for HTML generation parameters in the SharePoint Online connector configuration file. These parameters let you set which fields have higher or lower impact on matches.

To set up a schema, follow the instructions in Create and register a schema. When you set up a schema:

To map the names of SharePoint content types to corresponding object definitions, the connector normalizes the content type names by excluding unsupported characters. For object definitions, the Cloud Search API supports only A-Z, a-z, and 0-9 as valid characters. For example, the content type "Announcements" maps to the object definition "Announcements". The content type "News Article" maps to "NewsArticle" (no space).
When the connector can't match an object definition with an object definition, the connector uses the fallback object type (itemMetadata.objectType). Learn more about metadata configuration parameters.
The connector maps SharePoint property names to property definitions.
When the connector encounters Microsoft Outlook .msg files as it indexes content, it overrides the content type for the files and indexes them as application/vnd.ms-outlook.

Known connector limitations

Each connector instance can index content from only a single site collection. To index multiple site collections, set up additional connector instances.
Memory consumption increases with the number of unique users and groups that you use in ACLs for each site collection.
Delete notifications are not instantaneous and it can take more than 4 hours for a connector to recognize that a user deleted content from the source repository.

System requirements

System requirements
Operating system	Windows Server 2016 Ubuntu Red Hat Enterprise Linux 5.0 SUSE Enterprise Linux 10 (64 bit)
Authentication	Live Authentication
Software	SharePoint Online environment Java JRE 1.8 installed on the computer that will run the Google Cloud Search SharePoint Online connector

Deploy the connector

Prerequisites

Create a Google Workspace private key, which contains your service account ID. To learn how to get a private key, go to Configure access to the Google Cloud Search API.
Your Google Workspace administrator must add a data source to search. Record the data source ID.
If the connector returns results based on ACLs (results aren't public), your Google Workspace administrator must create two identity sources and give you their IDs:
- one for Office 365 users and groups
- one for SharePoint groups
The admin must also get your organization's Google Workspace customer ID and give it to you.

Learn how to get these values in Map user identities in Cloud Search.
Set up a user account for the connector that has SharePoint Online Site Collection Administrator privileges.
If the connector returns results based on ACLs (results aren't public), create credentials to read users and groups from your Office 365 account. Follow Microsoft's instructions to create an Azure Active Directory application. As you set up the app:
1. Name the app with a descriptive name, such as "gcs-o365-identity-connector".
2. Add permissions. The app needs "Group.Read.All" and "User.Read.All".
3. Record the following information to use in the connector configuration:
  - Application ID
  - Tenant
  - Client Secret

Step 1. Install the SharePoint Online connector software

Clone the connector repository from GitHub.

$ git clone https://github.com/google-cloudsearch/sharepoint-connector.git
$ cd sharepoint-connector

Check out the desired version of the connector:
```
$ git checkout tags/v1-0.0.3
```
Build the connector.
```
$ mvn package
```
To skip tests when you build the connector, run mvn package -DskipTests instead of mvn package.

Copy the connector zip file to your local installation directory:

$ cp target/google-cloudsearch-sharepoint-connector-v1-0.0.3.zip installation-dir
$ cd installation-dir
$ unzip google-cloudsearch-sharepoint-connector-v1-0.0.3.zip
$ cd google-cloudsearch-sharepoint-connector-v1-0.0.3

Step 2. Create the SharePoint Online connector configuration file

In the same directory as the connector installation, create a new file and name it sharepoint-online-connector.config. If you plan to run many connector instances, add more details to the name to distinguish it.

Add parameters as key/value pairs to the file contents, as in the following example:

### SharePoint Online connector configuration ###

# Required parameters for data source access
api.sourceId=08ef8becd116faa4546b8ca2c84b2879
api.serviceAccountPrivateKeyFile=service_account.json

# Required parameters for SharePoint Online access
sharepoint.server=https://mydomain.onmicrosoft.com
sharepoint.siteCollectionOnly=true
sharepoint.username=admin@mydomain.onmicrosoft.com
sharepoint.password=pa$sw0rd
sharepoint.formsAuthenticationMode=LIVE
sharepoint.deploymentType=ONLINE

# SharePoint identity mapping parameters
api.identitySourceId=08ef8becd116faa475de26d9b291fed9
api.referenceIdentitySources=defaultIdentitySource
api.referenceIdentitySource.defaultIdentitySource.id=08ef8becd116faa5d3783f8c5a80e5aa

# Optional SharePoint List Item metadata parameters
contentTemplate.sharepointItem.title=Title
contentTemplate.sharepointItem.unmappedColumnsMode=APPEND

For detailed descriptions of each parameter, go to the configuration parameters reference.

(Optional) Configure additional connector parameters, as needed. For details, go to Google-supplied connector parameters.

Step 3. Set up logging for the SharePoint Online connector

Create a folder named logs in the same directory that contains the connector binary.

Create a Latin-1-encoded file named logging.properties in the same directory and add the following content:

handlers = java.util.logging.ConsoleHandler,java.util.logging.FileHandler
# Default log level
.level = INFO
# uncomment line below to increase logging level for SharePoint APIsa
#com.google.enterprise.cloudsearch.sharepoint.level=FINE

# uncomment line below to increase logging level to enable API trace
#com.google.api.client.http.level = FINE
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.FileHandler.pattern=logs/connector-sharepoint.%g.log
java.util.logging.FileHandler.limit=10485760
java.util.logging.FileHandler.count=10
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

Step 4. Configure the SharePoint Online identity connector

This step is required to apply SharePoint Online identity-based ACLs to search results. If you set up the connector with public ACLs, you can skip this step.

In the same directory as the SharePoint Online connector installation, create a new file and name it sharepoint-online-identity-connector.config.

Add parameters as key/value pairs to the file contents, as in the following example:

### SharePoint Online identity connector configuration ###
# Required parameters for data source access
api.customerId=C05d3djk8
api.serviceAccountPrivateKeyFile=service_account.json

# Required parameters for SharePoint Online access
sharepoint.server=https://mydomain.onmicrosoft.com
sharepoint.siteCollectionOnly=true
sharepoint.username=admin@mydomain.onmicrosoft.com
sharepoint.password=pa$sw0rd
sharepoint.formsAuthenticationMode=LIVE
sharepoint.deploymentType=ONLINE

# SharePoint identity mapping parameters
api.identitySourceId=08ef8becd116faa475de26d9b291fed9
api.referenceIdentitySources=defaultIdentitySource
api.referenceIdentitySource.defaultIdentitySource.id=08ef8becd116faa5d3783f8c5a80e5aa

The values are almost the same as for the SharePoint Online connector, except that instead of api.sourceId, the parameter is api.customerId. The value of api.customerId is the customer ID that you got from your Google Workspace admin.

Step 5. Install the Office 365 Identity connector

This step is required to apply SharePoint Online identity-based ACLs to search results. If you set up the connector with public ACLs, you can skip this step.

Clone the connector repository from GitHub.

$ git clone https://github.com/google-cloudsearch/office-365-connector.git
$ cd office-365-connector

Check out the desired version of the connector:
```
$ git checkout tags/v1-0.0.3
```
Build the connector.
```
$ mvn package
```
To skip tests when you build the connector, run mvn package -DskipTests instead of mvn package.

Copy the connector zip file to your local installation directory:

$ cp target/google-cloudsearch-office-365-connector-v1-0.0.3.zip installation-dir
$ cd installation-dir
$ unzip google-cloudsearch-office-365-connector-v1-0.0.3.zip
$ cd google-cloudsearch-office-365-connector-v1-0.0.3

Step 6. Configure the Office 365 Identity connector and set up logging

In the same directory as the identity connector installation, create a new file and name it o365-identity-connector.config.

Add parameters as key/value pairs to the file contents, as in the following example. Note: All parameters are required.

### Office 365 Identity connector configuration ###

api.identitySourceId=12345abcde
api.customerId=abcde12345
o365.clientId=a63c6eb3-29e7-486
o365.tenant=185ef9ed-29e7-486
o365.clientSecret=raHJN15vRLBKs

For detailed descriptions of each parameter, go to the Office 365 Identity connector configuration reference.

Create a folder named logs in the same directory that contains the connector binary.

Create an ASCII or UTF-8 file named logging.properties in the same directory and add the following content:

handlers = java.util.logging.ConsoleHandler,java.util.logging.FileHandler
# Default log level
.level = INFO
# uncomment line below to increase logging level for Office 365 APIs
#com.google.enterprise.cloudsearch.o365.level=FINE

# uncomment line below to increase logging level to enable Google API traces
#com.google.api.client.http.level = FINE
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.FileHandler.pattern=logs/connector-o365.%g.log
java.util.logging.FileHandler.limit=10485760
java.util.logging.FileHandler.count=10
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

Step 7. Launch the SharePoint Online connector

In the following steps, you map the principals in both Office 365 and the SharePoint site collection to identities in the Cloud Identity service. This synchronization is done with the Office 365 Identity connector and the SharePoint Online identity connector.

After the Office 365 connector synchronizes the users and groups, run the SharePoint Online identity connector, to synchronize the SharePoint site collection groups. Then you run the SharePoint Online connector to index and serve results to your Cloud Search users.

Run the Office 365 identity connector:

$ java -Djava.util.logging.config.file=logging.properties -jar google-cloudsearch-office-365-connector-v1-0.0.3.jar -Dconfig=o365-identity-connector.config

Run the SharePoint Online identity connector:

$ java -Djava.util.logging.config.file=logging.properties -cp "google-cloudsearch-sharepoint-connector-v1-0.0.3.jar" com.google.enterprise.cloudsearch.sharepoint.SharePointIdentityConnector -Dconfig=sharepoint-online-identity-connector.config

Run the SharePoint Online connector:

$ java -Djava.util.logging.config.file=logging.properties -jar google-cloudsearch-sharepoint-connector-v1-0.0.3.jar -Dconfig=sharepoint-online-connector.config

Configuration parameters reference

Data source access

Setting	Parameter
Data source ID	`api.sourceId=1234567890abcdef` Required. The Google Cloud Search data source ID set up by the Google Workspace administrator.
Path to the service account private key file	`api.serviceAccountPrivateKeyFile=PrivateKey.json` Required. The path to the Google Cloud Search service account key file.

SharePoint Online access

Setting	Parameter
Fully-qualified domain name for the SharePoint Site Collection	`sharepoint.server=``http://yoursharepoint.example.com/` Required. If the domain name is not fully-qualified, then set DNS override on the connector host.
Site Collection Only Mode	`sharepoint.siteCollectionOnly=true` Required. For SharePoint Online, set the value `true`.
SharePoint username	`sharepoint.username=username` Required. The username for the account used to access SharePoint Online.
SharePoint password	`sharepoint.password=user_password` Required. The password for the account used to access SharePoint Online.
Authentication Mode	`sharepoint.formsAuthenticationMode=LIVE` Required. For SharePoint Online, the value is always `LIVE`.
Deployment Type	`sharepoint.deploymentType=ONLINE` Required. For SharePoint Online, the value is always `ONLINE`.

SharePoint Identity Mapping

Setting	Parameter
Identity Source ID	`api.identitySourceId=1234567890abcdef` Required. The Cloud Search identity source ID for the SharePoint groups identity source.
Reference Identity Sources	`api.referenceIdentitySources=defaultIdentitySource` Required. For sharePoint Online, the value is always `defaultIdentitySource`.
Reference Identity Source IDs	`api.referenceIdentitySource.defaultIdentitySource.id=112233abcd` Required. The Cloud Search identity source ID for Office 365 Users and Groups identity source.

HTML content generation

Setting	Parameter
HTML template title field	`contentTemplate.sharePointItem.title=Title` The SharePoint field to use as the HTML template title for generated HTML.
HTML content high search quality fields	`contentTemplate.sharePointItem.quality.high=highField1[,highField2,...]` A comma-separated list of fields to include in the generated HTML as high-quality fields. When the search query terms match these fields, the results are ranked higher.
HTML content medium search quality fields	`contentTemplate.sharePointItem.quality.medium=mediumField1[,mediumField2,...]` A comma-separated list of fields to include in the generated HTML as medium-quality fields.
HTML content low search quality fields	`contentTemplate.sharePointItem.quality.low=lowField1[,lowField2,...]` A comma-separated list of fields to include in the generated HTML as low-quality fields.
HTML content unmapped columns	`contentTemplate.sharepointItem.unmappedColumnsMode=APPEND` How the connector handles unmapped columns. Value is APPEND (default) or IGNORE. APPEND—The connector generates HTML content with all fields, including those that aren't set with a quality level (high, medium, or low). IGNORE—The connector generates HTML content with only mapped fields.

Office 365 Identity connector configuration

Setting	Parameter
Identity Source ID	`api.identitySourceId=1234567890abcdef` Required. The Office 365 identity source ID. This value should match the value of `api.referenceIdentitySource.defaultIdentitySource.id` in the SharePoint Online connector configuration.
Google Customer Id	`api.customerId=c1b1d1e1` Required. Your organization's Google Workspace customer ID.
Office 365 Application Id	`o365.clientId=a63c6eb3-29e7-486` Required. The application id from the Office 365 application setup.
Office 365 Tenant	`o365.tenant=185ef9ed-29e7-486` Required. The tenant for your Office 365 account (your Azure Activity Directory app)
Office 365 client secret	`o365.clientSecret=raHJN15vRLBKs` Required. Credential secret from Office 365 Application setup