Building a SharePoint App as a Timer Job
This post will show how to create an app as a timer job.
Background
One of the complicated parts of the app model today is trying to figure out how to do things that I used to do in full trust code using the app model. Honestly, things look a little different, and this pattern will be useful to understand.
As usual, if you aren’t interested in the narrative, skip down to the section, “Show Me the Code!”
I worked with a few customers who were concerned about some of their end users who kept using the new Change the Look feature of SharePoint 2013 to change the branding of their site. They turned their site into something hideous like this.
This doesn’t conform to their corporate branding, so they wanted a way to go back and change this in an automated fashion. Further, they want to do this on a daily basis to make sure the site is always changed back. They didn’t want to remove permissions for the user to do this. I’ll admit, there are other ways to do this, but it helps me to illustrate using a timer job to achieve the same thing.
Create the Console Application
In Visual Studio 2013, create a new Console Application. Here’s one of my favorite parts… go to the NuGet Package Manager and search for “sharepoint app”. You will see the App for SharePoint Web Toolkit.
Add this NuGet package to your Console app and you will get the TokenHelper and SharePointContext code to work with SharePoint apps.
Creating the AppPrincipal
The first step to understand is how to create the app principal. The app principal is an actual principal in SharePoint 2013 for the app that can be granted permissions. When you create an app in Visual Studio 2013 and press F5, Visual Studio is nice enough to take care of registering the app principal for you behind the scenes. It does this when using the Visual Studio template, but we’re not using their template here… we are using a Console Application. We need to register the app principal first for our Console Application to be able to call SharePoint.
To register the app principal, we can use a page in SharePoint 2013 called “_layouts/AppRegNew.aspx”.
This page is used to create the client ID and client secret for the app. Give it a name and click Generate, then click Create.
Note that I removed the actual string for the client secret for security purposes (hey, it’s a secret!)
The result is an app principal.
No, that is not the real client secret… I changed it for security purposes. Just use the string that the page generates and don’t change it.
Giving the App Principal Permissions
Now we need to grant permissions to the app principal. The easiest way to do this is to create a new provider-hosted app in SharePoint, give it the permissions that your app needs, then go to the appmanifest.xml and copy the AppPermissionRequests element.
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="https://sharepoint/content/tenant" Right="Manage" />
</AppPermissionRequests>
The permissions that we will grant will be Tenant/Manage permission because our Console Application will go to multiple webs that are located in multiple site collections and change the branding. To have permission to access multiple site collections, I need to request Tenant permission. To change the branding for a site, I need Manage… hence Tenant/Manage.
You then go to a second page in SharePoint called “_layouts/AppInv.aspx”.
Look up the app based on the Client ID that you just generated and click Lookup, it will find the app principal. Then paste the AppPermissionRequests XML into the Permissions text box and click Create.
Once you click Create, the result is the Trust It dialog.
Click Trust It (of course you trust it).
App Only Permission
I previously wrote a blog post, SharePoint 2013 App Only Policy Made Easy, that talks about the app only policy. If you aren’t familiar with app only, you need to go read that post. Our timer job will not have an interactive user, so we need to use the app only policy. The relevant code for this is the TokenHelper.GetAppOnlyAccessToken.
//Get the realm for the URL
string realm = TokenHelper.GetRealmFromTargetUrl(siteUri);
//Get the access token for the URL.
// Requires this app to be registered with the tenant
string accessToken = TokenHelper.GetAppOnlyAccessToken(
TokenHelper.SharePointPrincipal,
siteUri.Authority, realm).AccessToken;
Once we have the access token, we can now create a ClientContext using that access token.
using(var clientContext =
TokenHelper.GetClientContextWithAccessToken(
siteUri.ToString(),accessToken))
We now have a client context to use with the rest of our CSOM operation calls.
Update the App.config
The app.config will be used to store the URLs for various webs that need to have their branding updated via a timer job. The app.config also stores the Client ID and Client Secret for our app.
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="Sites"
type="System.Configuration.NameValueSectionHandler"/>
</configSections>
<appSettings>
<add key="ClientId" value="0c5579cd-c3c7-458c-91e4-8a557c33fc50"/>
<add key="ClientSecret" value="925gRemovedForSecurityReasons="/>
</appSettings>
<startup>
<supportedRuntime version="v4.0"
sku=".NETFramework,Version=v4.5" />
</startup>
<Sites>
<add key="site2"
value="https://kirke.sharepoint.com/sites/dev"/>
<add key="site1"
value="https://kirke.sharepoint.com/sites/developer"/>
<add key="site3"
value="https://kirke.sharepoint.com/sites/dev2"/>
</Sites>
</configuration>
Our Console Application will read the Sites section, pull the URL for each site, and call CSOM on it to update the branding.
Show Me the Code!
using Microsoft.SharePoint.Client;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace TimerJobAsAnApp
{
class Program
{
/// <summary>
/// To register the app:
/// 1) Go to appregnew.aspx to create the client ID and client secret
/// 2) Copy the client ID and client secret to app.config
/// 3) Go to appinv.aspx to lookup by client ID and add permission XML below
/// </summary>
/// <param name="args"></param>
/*
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="https://sharepoint/content/tenant" Right="Manage" />
</AppPermissionRequests>
*/
static void Main(string[] args)
{
var config = (NameValueCollection)ConfigurationManager.GetSection("Sites");
foreach (var key in config.Keys)
{
Uri siteUri = new Uri(config.GetValues(key as string)[0]);
//Get the realm for the URL
string realm = TokenHelper.GetRealmFromTargetUrl(siteUri);
//Get the access token for the URL.
// Requires this app to be registered with the tenant
string accessToken = TokenHelper.GetAppOnlyAccessToken(
TokenHelper.SharePointPrincipal,
siteUri.Authority, realm).AccessToken;
//Get client context with access token
using(var clientContext =
TokenHelper.GetClientContextWithAccessToken(
siteUri.ToString(),accessToken))
{
//Poor man's timer
do
{
ApplyTheme(clientContext);
System.Threading.Thread.Sleep(10000);
}
while (true);
}
}
}
/// <summary>
/// Applies a red and black theme with a Georgia font to the Web
/// </summary>
/// <param name="clientContext"></param>
private static void ApplyTheme(ClientContext clientContext)
{
Web currentWeb = clientContext.Web;
clientContext.Load(currentWeb);
clientContext.ExecuteQuery();
//Apply RED theme with Georgia font
currentWeb.ApplyTheme(
URLCombine(
currentWeb.ServerRelativeUrl,
"/_catalogs/theme/15/palette022.spcolor"),
URLCombine(
currentWeb.ServerRelativeUrl,
"/_catalogs/theme/15/fontscheme002.spfont"),
null, false);
clientContext.ExecuteQuery();
}
private static string URLCombine(string baseUrl, string relativeUrl)
{
if (baseUrl.Length == 0)
return relativeUrl;
if (relativeUrl.Length == 0)
return baseUrl;
return string.Format("{0}/{1}",
baseUrl.TrimEnd(new char[] { '/', '\\' }),
relativeUrl.TrimStart(new char[] { '/', '\\' }));
}
}
}
The Result
The sites in the app.config used to have that hideous theme. Once the code runs, the sites in the app.config will have the colors of my favorite college football team (Go Georgia Bulldogs!), even using the “Georgia” font 
Of course, you can use whatever logic you want, the logic we use here is setting branding based on pre-configured URLs for the sites. You can use whatever you want to schedule the timer job. I used a poor man’s timer job, using a While loop with Thread.Sleep, but you could use the Windows Scheduler, a Cron job, or event Azure Web Jobs.
For More Information
SharePoint 2013 App Only Policy Made Easy
Comments
Anonymous
March 03, 2014
Thank you! Very interesting for me technical approach to combine the app model and console application.Anonymous
March 25, 2014
Thanks for the info. I have a query, Can I develop this scenario with Visual studio 2012?Anonymous
March 26, 2014
@Vardhini - absolutely. Use the NuGet package manager to add the App for SharePoint Web Toolkit for the Console application and that's all you need.Anonymous
March 30, 2014
The comment has been removedAnonymous
March 30, 2014
Vardhini - If I had to guess, it sounds like you did not go to AppRegNew.aspx to create the client ID and client secret yet in your SharePoint site and then replace the values in the sample code with your own client ID and client secret. Also double-check that you are using the URL for your SharePoint Online site and not "kirke.sharepoint.com".Anonymous
March 31, 2014
The comment has been removedAnonymous
April 01, 2014
Varhhini - what values are you using for the app.config? Make sure to replace these values with the path to your web:<Sites> <add key="site2" value="kirke.sharepoint.com/.../> <add key="site1" value="kirke.sharepoint.com/.../> <add key="site3" value="kirke.sharepoint.com/.../> </Sites>Anonymous
April 01, 2014
I replaced this at the very first time I created the application. I ensured that on first hand. Again I verified when you asked me to. Still the problem persist. Sorry for bugging you many times :(Anonymous
April 02, 2014
Varhhini - I wish that I could provide better guidance, but the blog post above works and I've used it with many customers. There's something we've not been able to discover about your environment, but I am not able to determine it based on our conversation. Perhaps post to stackoverflow.com/.../sharepoint as the product team, MVPs, and many community members monitor that regularly... you have a much higher chance of getting further assistance there from a broader set of people. I apologize that I cannot solve your issue, I'm not sure what your problem might be.Anonymous
April 08, 2014
Thanks for your suggestion. I will post my query as you saidAnonymous
April 10, 2014
Kirk, for creating a clientcontext we normally need the sitecollection url, right? however one scenario I have is that I have 20,000 site collections in one web application, with server side you can normally iterate over all sites in a web application, can this be done with a console app/csom?Anonymous
April 10, 2014
Luis - There is an example in our OfficeAMS project that shows an approach to iterating site collections. http://officeams.codeplex.com/Anonymous
April 11, 2014
Thanks Kirk, I wanted to build your example but based on a different scenario, I want to run a timer job that searches for new files and then sends an email, I have followed all your steps but I get an unauthorized exception,I have asked the question here:stackoverflow.com/.../rest-search-api-the-remote-server-returned-an-error-401-unauthorizedI hope you or somebody from your team might take a look at it.thank youAnonymous
May 04, 2014
The comment has been removedAnonymous
May 06, 2014
Hi Kirk,It was good to meet you are Techfest! I was using an Azure webjobs to run an "unattended" sharepoint app like this... but I was using SharePoinOnlineCredentials with ClientContext.Obviously I do not want to deal with raw passwords, so I'll try to implement this.I'd like to understand more about what exactly TokenHelper is doing in this scenario. For example: to get the Realm I notice that TokenHelper sends a request to /_vti_bin/client.svc with a header "Authorization: Bearer"Is there a good article where I can read more about client.svc and what it's doing exactly?Anonymous
May 06, 2014
Client.svc is the endpoint for the client side object model and the REST API, both call that same endpoint. Here is a good starting point to dive into everything about CSOM, REST, OAuth 2.0, and the app model.msdn.microsoft.com/.../jj163794(v=office.15).aspxAnonymous
May 20, 2014
@VardhiniI had the same issue with message "The requested namespace does not exist". The problem is, that the authentication uses ACS by default. If ACS ist not configured properly, the access to the ACS provider does not work. Using Fiddler you can see how the program is trying to access the ACS provider:GET accounts.accesscontrol.windows.net/.../1The ACS provider will return the error message "The requested namespace does not exist".In my environment I use STS. You have to configure STS (create a *.pfx certificate, register an issuer with New-SPTrustedSecurityTokenIssuer) and add the following keys to your app.config<add key="ClientSigningCertificatePath" value="....." /><add key="ClientSigningCertificatePassword" value="....." /><add key="IssuerId" value="....." />Afterwards you can create a ClientContext the following way:bool isHighTrust = TokenHelper.IsHighTrustApp();ClientContext clientContext = null;if (isHighTrust){ WindowsIdentity windowsIdentity = WindowsIdentity.GetCurrent(); ClientContext clientContext = TokenHelper.GetS2SClientContextWithWindowsIdentity(siteUri, windowsIdentity); ....}Now I could access the SharePoint data.Anonymous
July 18, 2014
The comment has been removedAnonymous
July 18, 2014
The comment has been removedAnonymous
July 18, 2014
Thanks for the quick reply, now its reading others comments as well.Anonymous
August 11, 2014
Nice article, I have developed an simple app, but when i tried to add it on the page ,my app is not showing. otherwise apps is working fine.Anonymous
August 20, 2014
kirk,Great article... but this code has a critical flaw.... you are manually entering each and every site collection in a config file. this will be a pain because the site admin will have to come on a daily basis and update the config file.the central limitationof the app model is that there is no way to iterate over the site collections inside of a farm / tenant (via the app model). this is trivial in the full trust world... but is not possible in the app world.How can you modify this app so that it dynamically determines each site collection URL instead of relying on a config file?Anonymous
August 20, 2014
@Abhishek - see the Core.SiteEnumeration example in the Office Dev/PnP project at github.com/.../PnP.Anonymous
September 11, 2014
Will this work in a multi-tenant scenario? Say I want to create a Web job as a part of a SharePoint app that can be installed to any tenant and all tenants will share the same Azure Web Job. How is Authentication handled in this case? And how do I know from what URL I should open my client context since I don't have the query string properties (i.e. host web) I get in a regular web app.Anonymous
September 11, 2014
Please feel free to delete my earlier comment.Will this work in a multi-tenant scenario? Say I want to create a Web job as a part of a SharePoint app that can be installed to any tenant and all tenants will share the same Azure Web Job. How is Authentication handled in this case? And how do I know from what URL I should open my client context since I don't have the query string properties (i.e. host web) I get in a regular web app.Anonymous
September 11, 2014
@Gabriel - absolutely, this would work for a multi-tenant scenario. You will just need to figure out the URLs that you are going to access for each tenant.Note that this solution is intentionally simple in its design. You would likely need to have multiple instances of your WebJob running in parallel, in which case you need additional patterns such as competing consumer. See the following page for some of the most frequent cloud architecture patterns:azure.microsoft.com/.../architecture-overviewAnonymous
September 14, 2014
Hello,I tried to run your example, using my own app that I registered. Does SharePoint 2013 needs to be connected with Azure Access Control Service ? I am asking this because when I try to get the acessToken from TokenHelper class I get this exception:The remote server returned an error: (404) Not Found. - The requested namespace does not exist.Here is how the url for accesscontrol looks in TokenHelper:accounts.accesscontrol.windows.net/.../1Anonymous
September 28, 2014
HeyThank you for a great post. I works perfect.However, I have a question. We need a timejob which is using search. But we are not allowed to use search with app only permissions because of the security trimming and the "QueryAsUserIgnoreApp" permission level. Is there a way to go around this so we can use app princiapals. I know we can solve this problem by using credentials(username and password) but we would prefer not us this method.Anonymous
September 28, 2014
Search doesn't work with app-only policy due to per-user security trimming. See blogs.msdn.com/.../what-every-developer-needs-to-know-about-sharepoint-apps-csom-and-anonymous-publishing-sites.aspx for some discussion on exposing anonymous content using the REST API.Anonymous
October 12, 2014
Hi Kirk,Very nice article.I successfully created my timerjob and now i want to have a webpage where i can configure the job. Do you have any experience or ideas on how I can write configuration from a Website in Azure and consume the configuration from a timerjob?Anonymous
November 04, 2014
Hi Kirk, brilliant article.I'm using this approach to create a timer job that list site users and depending of a user profile property send an email to the user. In order to accomplish that I create a PeopleManager object passing the ClientContext and then I execute a query, when I do that I obtain the following exception:Microsoft.SharePoint.Client.ServerException: User 'i:0i.t|00000003-0000-0ff1-ce00-000000000000|app@sharepoint' doesn't exist in UPA by UPN or SID, and user withthis SID was not found in AD. at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream) at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse() at Microsoft.SharePoint.Client.ClientRequest.ExecuteQueryToServer(ChunkStringBuilder sb) at Microsoft.SharePoint.Client.ClientRequest.ExecuteQuery() at Microsoft.SharePoint.Client.ClientRuntimeContext.ExecuteQuery() at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery() at Global.Spotlight.EndProbationTimerJob.Program.Main(String[] args)Here is my code://Get client context with access token using (var clientContext = TokenHelper.GetClientContextWithAccessToken(siteUri.ToString(), accessToken)) { var siteUsers = clientContext.Web.SiteUsers; clientContext.Load(siteUsers, users => users.Include(user => user.Title, user => user.LoginName)); clientContext.ExecuteQuery(); foreach (var user in siteUsers) { var peopleManager = new PeopleManager(clientContext); var properties = peopleManager.GetPropertiesFor(user.LoginName); clientContext.Load(properties, x => x.UserProfileProperties); clientContext.ExecuteQuery(); } }Do you think there is another way to do it or it is just impossible?Anonymous
November 20, 2014
Hi Kirk,We are having exact same problem with Jordi above, all other staging tenant was working fine except production, wondering can you give us some clue where it might go wrong?we use apponly permission with tenant perm to access user profile service on sharepoint online.thanks,Anonymous
November 20, 2014
Apologies, but I don't have the personal bandwidth to help troubleshoot. This worked at the time of its writing. Support resources include social.msdn.microsoft.com/.../home and stackoverflow.com/.../sharepoint. I wish that I had more time to help.Anonymous
December 08, 2014
Hi Kirk,Nice article and it is very helpfulBut We got the same issue like Jordi, Eric as they mentioned above.@Jordi,@Eric : Have you figured it out the issue ?Anonymous
December 21, 2014
Hi Vardhini, I guess you are using On Premise host for that example. If yes you should add a few keys in your app.config <add key="ClientSigningCertificatePath" value="..." /> <add key="ClientSigningCertificatePassword" value="..." /> <add key="IssuerId" value="..." />And after that try to use string accessToken = TokenHelper.GetS2SAccessTokenWithWindowsIdentity(siteUri, null);ThanksAnonymous
January 15, 2015
Hi Jordi and SRam,When you are modifying or accessing other people's user profiles, you should be accessing the tenant admin URL Here's good example from the Office 365 Developer Patterns and Practices for that - github.com/.../UserProfile.Manipulation.CSOM.Console.If you have any questions, please use the Office 365 Dev PnP Yammer group at aka.ms/officedevpnpyammer. We have more than 1500 others in this group helping each other with specific technical questions towards app model development with SharePoint or Office 365.Anonymous
May 04, 2015
Hi Kirk,I went through the steps explained, facing 401 unauthorized exception GetRealmFromTargetUrl function.Will you guide further ? thanksAnonymous
May 29, 2015
The comment has been removedAnonymous
June 01, 2015
Hi Kirk, I have created a Azure web jobs to provision the site collection remotely. I have registered the app in SPO with Tenant, Taxonomy, Site Collection & Site with Full control privilege. My Azure job is creating the site collection successfully but am unable to create a Term in Term store. I am getting an error - "Access denied. you don't have permission to access or read this resource" :( I am using Term store for Custom Global Navigation for across site collections. Can you please help me on this?Anonymous
July 20, 2015
Hello Kirk, I am trying to create a Sharepoint Provider Hosted App, I need to configure timer job on a list created in my app. So what would be the web url for the below code: <Sites> <add key="site2" value=""/> </Sites> Is it contain App-Web url or Client's web Url ? And How can I configure this console App when my SP app will be on Server ?Anonymous
October 14, 2015
Hello everybody, When working with SharePoint Designer Workflow on SharePoint Online (Office 365) and trying to access to User Profile Service (UPS) make sure you follow the instructions on the following post by providing the Workflow App the necessary permission to the UPS, but make sure you don't run the REST call in an App Step, this caused me a lot of Troubleshooting time as I was getting the error mentioned above "...app@sharepoint doesn't exist in UPA by UPN or SID, and user with this SID was not found in AD." sharepoint-community.net/.../retrieving-user-profile-properties-in-a-sharepoint-2013-workflow I hope it helps you too. Have fun.Anonymous
January 04, 2016
very usefull, thank you!- Anonymous
June 19, 2016
Hi Kirk,https://blogs.msdn.microsoft.com/webdev/2014/11/12/new-developer-and-debugging-features-for-azure-webjobs-in-visual-studio/Can we configure this webjob as SharePoint App as instead of creating Separate console app and then configuring it in WebApp?Regards,Swati
- Anonymous
Anonymous
June 19, 2016
Hi Kirk,https://blogs.msdn.microsoft.com/webdev/2014/11/12/new-developer-and-debugging-features-for-azure-webjobs-in-visual-studio/Can we configure this webjob as SharePoint App as instead of creating Separate console app and then configuring it in WebApp?Regards,Swati