Gae Python - How To Set A Cron Job To Launch A Backend Task

I'm running a daily reporting task on GAE which since recently is using too much memory to finish. Therefore I'd like to set it as a backend task. I've set the backend as following

Solution 1:

Depends if you want a persistent or dynamic backend

For a dynamic one

The plan is:

1. A cron fires at specific time.

2. Adds a task on a queue that will start the backend

3. The backend starts

Example:

app.yaml:

-url:/crons/startgooglepluscrawler/script:crons.startgooglepluscrawler.applogin:admin

backends.yaml:

backends:-name:google-plus-crawlerclass:B2start:backends.googlepluscrawler.appoptions:dynamic,failfastinstances:1

crons.yaml:

cron:
- description: get daily google plus user followers and followings
  url: /crons/startgooglepluscrawler/
  schedule: everyday09:00

queue.yaml:

total_storage_limit:10Mqueue:-name:google-plus-daily-crawlingrate:1/sretry_parameters:task_retry_limit:0task_age_limit:1s

On the startgooglepluscrawler.app you need to start the backend with a taskqueue:

classStartGooglePlusCrawlerHandler(webapp2.RequestHandler):

    defget(self):
        logging.info("Running daily Cron")
        taskqueue.add(queue_name = "google-plus-daily-crawling",
                    url="/_ah/start",
                    method='GET',
                    target=(Noneif self.is_dev_server() else'google-plus-crawler'),
                    headers={"X-AppEngine-FailFast":"true"}
                    )
        logging.info("Daily Cron finished")

    defis_dev_server(self):
        return os.environ['SERVER_SOFTWARE'].startswith('Dev')


app = webapp2.WSGIApplication([
        ("/crons/startgooglepluscrawler/",StartGooglePlusCrawlerHandler)

    ],debug=True)

And at the backends/googlepluscrawler.py just normally like an app, and a handler to /_ah/start:

app = webapp2.WSGIApplication(
            [('/_ah/start', StartHandler)],
            debug=True,
            config=config.config)

The above example will fire up the backend instance.

Solution 2:

An easier way to do this is by migrating the app to modules. Explained here: https://developers.google.com/appengine/docs/python/modules/

After doing so, you can just add following line in the cron.yaml:

target: yourmodule

This allows the cron job to run on the instance defined in yourmodule.yaml