Chris Pollett > Students >
Padia
    ( Print View)

    [Bio]

    [Blog]

    [CS297 Proposal]

    [CS 297 Report PDF]

    [CS298 Proposal]

    [CS298 Report PDF]

    [CS298 Oral Defence Slides PDF]

    [Deliverable 1: CacheRefresh MediaJob]

    [Deliverable 2: Implement MLDC Algorithm]

    [Deliverable 3: Implement STDC Algorithm]

    [Deliverable 4: Implement SSDC Algorithm]

    [Understanding Yioop PDF]

    [Scalability Challenges PDF]

    [Cache Aware strategies PDF]

    [ML Based Cache Algorithm PDF]

    [Static Topic Dynamic Cache PDF]

    [Static Semi-Static Dynamic Cache PDF]

    [Query Statistics]
Deliverable 1
Implement CacheRefresh Media Job

Purpose
The goal of this deliverable was to get familiar with Yioop code base, analyze important classes and methods related to Media Job. Along with this, implement a new Media Job to refresh cache with popular queries.

Description
Currently Yioop has only a dynamic cache which is populated with popular queries offline during the restart. To refresh the cache, a script has to be run manually. The top most popular queries have been empirically decided and are based on past data. The dynamic contents are updated in cache with execution of every search query.
To periodically refresh the cache, a Media Job has been created. Media jobs are the processes that are scheduled to run periodically by Media Updater of Yioop. Having cache refreshed automatically and periodically ensured that the cache is up to date with popular queries. Code: CacheRefresh Media Job

Conclusion
We analyzed the important classes and methods related to Media Jobs. Also, we have implemented a Media Job to refresh the cache which will be helpful in the next semester to implement any algorithm which require static cache to be populated periodically

Classes Description
MediaJob It is a parent class for the Media Job to be executed. All jobs need to extend this class and override callback methods to execute their intended functionality
MediaUpdater It is a class reponsible for executing and managing lifecycle of Media Jobs

Methods Description
init() Callback method called after constructor to initialize the job.
checkPrerequisites() Method that should returns true if media job has to be executed, false otherwise.
nondistributedTasks() Method is called when Media Job is running in non-distributed mode. Non-distributed version on task is can be added in this method
prepareTasks() Method is responsible for preparing data for the tasks executed on client machines in distributed mode. The method is executed on the NameServer only.
getTasks() Client machine call this method on NameServer to get their data. This method uses output of prepareTask() method and send client its indiavidual share of the data.
doTasks() The method to process the data fetched from getTasks() method. This is called by MediaUpdater on each client after they complete getTasks().
putTasks() Each client call this method on the NameServer to store their processed results on the NameServer
finishTasks() The method is called on the NameServer to complete the final computation after all tasks are completed