WEEKLY PROJECT PLAN Changes since the last update have been highlighted in red. This week’s update: WEEKLY UPDATES 9/25 – After several weeks of resolving bugs and issues, the phase one prototype for the digital library is now live, at mdic.force.com. At this point, there are still a few problems to solve, and we hope to resolve these and have the site ready for users soon. We are testing the site with users, groups, submissions, search, and metadata. During the testing process we uncovered a problem with submissions entering the peer review process. This problem is hopefully a simple adjustment, and then the process can be tested more thoroughly. The collaborative groups are still being tested for sharing files and privacy settings. PROJECT DESCRIPTION State what the project will achieve in twenty-­‐five words or less. Be clear and concise. Use plain language. Create a vision. This project will create a digital library for computer modeling and simulation that is focused on sharing quality materials. Phase 1 is released to MDIC project teams, and phase 2 is public release. Data in the library will be curated and stored in tiers. These collections of data can be rated, reviewed, searched and easily downloaded. Groups can collaborate and communicate privately while completing MDIC projects and publish deliverables publicly. STRATEGIC ALIGNMENT Describe how the project fits within the strategic goals of the company – the project sponsor should provide the majority of this section based on business priorities. Includes Nature of Project Working groups create content, and MDIC is where the content is shared. The library allows a place to showcase that content. MDIC work should be trusted, credible and reproducible. As an organization, MDIC is associated with the creation and use of regulatory grade models and data. OBJECTIVES List the three to five project outcomes that will define the project success: Each objective must be specific and measurable. FDA Library Concepts Knowledge Transfer October 2014 Phase 0: Prototype large file upload/download/access; March 2015 Phase 1: Be the one place all MDIC CM&S work is archived, models and data shared This is for the working groups, not yet public August 2015 Phase 2: Store and curate regulatory grade models and data TBD Based on Resources Public Deployment: Sustainable model for funds and resources TBD Based on Resources KEY SUCCESS FACTORS List the prioritized list of how success will be determined – e.g. full feature set, meeting market window, coming in at cost Phase 0: All files sizes and types can be uploaded. Phase 1: Access to files and group space is restricted appropriately. Searching is robust and accurate, covers all relevant data, has faceted searching. Infrastructure for submissions to enter and proceed through the review process. MDIC working groups are satisfied with functionality. Phase 2: Review process is staffed, sustainable plan for funding and maintaining infrastructure. Gain a high degree of ongoing community engagement, resulting in model and data contribution, review, and use. WHAT IT IS AND WHAT IT IS NOT Often it is as important to define what the project will not include – this prevents missed expectations. § § § § § § § § § WHAT’S IN MDIC is building library infrastructure for models and data Using Salesforce for library platform Files will be stored in Amazon Metadata based searching, filtering and browsing Curate the data – review process, community feedback; rate and comment Open to the public (in phase 2) Uploading any file size (<TB) or type as a collection, accessed through metadata. Approved users can share without review Relevant and trustworthy source for computer modeling and simulation § WHAT’S OUT FDA not hosting an external library § § § Open source or custom built code Storing files in Salesforce Tags (in phase 1) § Private storage for individual users § § Unapproved groups Downloading without registering A completely open file sharing “wild west” § A library for all knowledge § CRITICAL RISK FACTORS List the key factors likely to constrain success of the project – e.g. technology, critical resource availability, or aggressive schedule. Outline how the risks will be mitigated. Needs a community to use and value the library. Sustainability plan for costs and staff associated with the library Infrastructure has to be reliable, meet the user’s needs. How will privacy settings keep files restricted to their tier/group? What searching capabilities will be implemented? INTERDEPENDENCIES Determine factors outside your project that could influence it. • How does the project align with other initiatives – is there overlap, areas for efficiencies? • Does another project have input to yours? Is a downstream project counting on your output? • Is there a technology deliverable? Internal or external? • Are there resource conflicts? Possibility of second library/community for PCBR? Ramp up time for getting data and models from the community Volunteer availability/commitment from all MDIC member stakeholders ASSUMPTIONS List key assumptions used to derive the analysis. List the source and how the assumption impacts this project. Salesforce platform is accessible for all users (depends on their IT policy in their workplace), the files can be uploaded to Amazon, through the use of DocuVault, search will occur on the metadata which return the collection of files. People will use and contribute data for the library. Scalable architecture: Phase 1 – MDIC users Phase 2 – expanded community, including the public KEY MILESTONES* DUE Assessment Concept Planning and Specification Development Controlled Release Commercial Release FDA Knowledge Transfer Phase 0 Deployment MDIC Board of Directors approval for Phase 1 funding Steering Committee Meeting on Phase 1 Working Group Phase 1 Meeting 1 Phase 1 Functional Prototype Working Group Phase 1 Meeting 2 Phase 1 Deployment MDIC Journal Publication Submitted MDIC Board of Directors Meeting MDIC Annual Meeting Working Group Work Packages 1-­‐15 First Model and Data Submission FY15 FY16 PROJECT COST Plan Actual Plan October 2014 March 2015 May 2015 7/15/15 July 2015 August 2015 August 2015 Q4 2015 Q4 2015 9/24/15 9/25/15 TBD TBD Actual FY17 Plan Actual Resources Capital Expense Phase 1 (Ongoing costs): RedPath Maintenance DocuVault Licenses • 1st 500 licenses at $2500 ($5/user/year) • Subsequent 500 user "packs" at an additional $500/pack/year ($1/user/year) Salesforce Community • $200/month for 2000 logins/month Amazon S3 Cloud Storage • .03/GB/month = $30/TB/month based on amount used, not capacity Phase 1 Example (Includes MDIC project teams only): 500 users for one year, 100 TB of data = 2,500 DocuVault 36,000 Amazon 2,400 SalesForce 6,000 RedPath -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ 46,900 total Phase 2: TBD If necessary: Salesforce Community Plus • $600/month for 2000 logins/month ($7200/year) TEAM At a minimum, list required functions (or responsibilities) in early phases. If a specific resource or skill is required, note this. Redpath is the consulting firm, performing customization of the Salesforce platform. Library working group: consulted on requirements and prototype feedback Library working group chair – Dave Flynn: accountable for the work packages Work package teams: responsible for work packages Testing Team: responsible for testing the prototype Steering Committee: informed about progress Project Leadership – Randy, Kyle: consulted about progress Program Manager – Dawn: accountable for overall progress MDIC Library Intern – Katie (phase 1): accountable for testing team, responsible for managing interface with contractor STATUS/DETAILS This section can be used to capture key status items, implementation details, or objectives that are too specific for the higher-­‐
level information. SalesForce has been selected as the platform, for their commercial off-­‐the-­‐shelf software. They offer nonprofits their product for 80% off the retail price, and offer a signal sign-­‐on for SalesForce and partner services like Amazon S3 cloud storage. The security of SalesForce and Amazon is enterprise level, and the data storage through Amazon S3 is easily integrated, durable, and highly scalable. The cost is based only on the space that is used. Additionally, the SalesForce system is fully auditable, and it is whitelisted at FDA. SalesForce allows customization 3rd party apps from a thriving community, and the software is FedRAMP approved. Further, we currently have over 100 MDIC members -­‐individuals who are currently accessing the MDIC SalesForce chatter for working group discussions and file sharing. SalesForce can be accessed through a log in from anywhere, and users never need to install any software. SalesForce Communities offers licenses as a monthly subscription fee that allows us pay for logins per month, instead of identifying and paying for all possible users. This option keeps the monthly costs low, and allows many different users to access the library. Communities are a recent addition to the SalesForce product line, mostly developed within the last year. Using SalesForce Communities requires us to push the boundaries of the platform, and this is accomplished through customization by a consulting firm – RedPath. What can be stored in the library? The library is designed to be extremely flexible. It can hold any type of file; items can be grouped together and uploaded as a collection. There are no limitations on the size of a file or collection, storing large models and simulations is a crucial point of the library. The library allows the community to link to outside sources for sharing related data. Where do the files come from? Submissions to the library can come from many sources. If a researcher or team, from academics or industry, is ready to publish a collection the library will offer a place for peer review and assign the files a DOI-­‐ digital object identifier. Members working in a group can collaborate and submit their project to the library. Additionally, the library will house regulatory grade content such as Medical Device Development Tools qualified FDA. The initial content will come from MDIC working groups. Who can use the library? Anyone who registers for the website can access the public shared files. These will be stored in a tiered system. Tier 0 will hold files private to the working groups; these are works in-­‐progress before submission to the peer review process. Tier 1 will hold files that are shared publicly, by approved members only, but are not peer reviewed. Tier 2 will hold files that have been peer reviewed. Finally, Tier 3 is the highest level, containing regulatory grade content. There is a possibility of having verified users of the library. This would require an identification/verification process to allow users to access certain data in the library. What should be included with a submission? Metadata will need to be submitted with the collection upload, certain fields are required and others are optional. The collection should include a pdf document for the review process; this will describe the data in the collection and may be used in the journal publication. Image files, such as jpg or png, can be uploaded as part of the collection to allow previewing the model or simulation in browser, these will assist the user in finding relevant files before downloading. WEEKLY UPDATES 9/25 – After several weeks of resolving bugs and issues, the phase one prototype for the digital library is now live, at mdic.force.com. At this point, there are still a few problems to solve, and we hope to resolve these and have the site ready for users soon. We are testing the site with users, groups, submissions, search, and metadata. During the testing process we uncovered a problem with submissions entering the peer review process. This problem is hopefully a simple adjustment, and then the process can be tested more thoroughly. The collaborative groups are still being tested for sharing files and privacy settings. 9/18 – Testing of the community is uncovering some issues and limitations. At the time of deployment, there were a few bugs in the site, and Redpath has worked to fix these. There were security issues with the groups and settings that needed to be adjusted internally for creating users. Further testing is being completed, test users are being registered and data is being uploaded and entered for submissions. 9/11 – The SalesForce community for phase 1 of the digital library has been deployed. At this point, it is early in testing, next week should give us a better idea of how the site is working and meeting our needs. 9/04 – The site has faced delays for the last week, there were necessary issues to resolve with the review process. Files will enter the review process, and will be visible to the review panel, but not visible to all users of the site during the time the submission is being reviewed. This level of security is being implemented throughout the site with the custom objects, and within the DocuVault app. The interactions between these items are causing complications, resulting in short delays. The search feature for phase 1 are expected to cover the metadata fields that accompany a submission to the library. Limited testing has shown that descriptions can be searched, and will allow for filtering. As next week begins, we expect more updates on the delays and timeline. 8/27 – The proof of concept is getting ready for deployment. Several changes to the security model have allowed posts to remain within the private groups. Final additions to the submissions upload have added the metadata fields. The deployment of the site has been delayed, due to complication in the security for the peer review process. 8/20 – During the last week, we got an idea of the how files can be uploaded as submissions, and then how the review process might operate. Yet, this initial testing of the proof of concept site has revealed several additional security issues. In the library we will have private working groups, and they will be separate from the Chatter found in SalesForce. To start testing features, we have started to enter test users and create collaborative groups in the development sandbox. This is how more privacy issues were uncovered, and the need to push the platform into new territory for collaboration. The licenses for SalesForce are based on standard Communities users, which offers limited customization for security. A possible solution exists with Communities Plus licenses, and this can be considered for the next phase, or if further development on this proof of concept uncovers more issues. 8/13 – This week we had a demo to give us a first look at the library while it is still in development. The previous complications with security models have been resolved and we can get an idea how the working groups will function. The ability to make public and private groups for works in progress was a key factor of this library. We now know that users can collaborate in a group space, and files can be restricted. The file upload for submissions allows us to test large files and collections, but the peer review process is still in development. In the next couple weeks, we should start to see final changes to the digital library and phase 1. 8/6 – Due to DocuVault and SalesForce Communities being fairly new applications, we are having complications with adjusting their security models to fit our needs. This has resulted in a delay in the release of the library. It is expected to be finished and ready for testing around the end of August. The first phase 1 library meeting was held on July 22, and a repeat session on July 28. After these meetings we had a call for volunteers to assist with work packages. The details for these packages, and other documents, can be found on the MDIC working group webpage: http://mdic.org/computer-­‐
modeling/wg-­‐cms-­‐library. Also, we are still in need of more people to volunteer and we will hold a work package meeting next week. Please send an email if you can help with any of the work packages, or testing the library prototype. 7/29 – A first look at the demo of DocuVault file upload. This app integration offers various click or drag and drop uploading, file privacy, the ability to grant limited time access. Next week’s demo will be based on the security models. NOTES FROM THE COMMUNITY Topics of discussion from emails and meetings, any additional community response and feedback can be added here. It is unfortunate that so many resources have been developed and remain underutilized and largely unknown (e.g., example, the Physiome project, the EU Heart project which is no longer funded but the database has been resurrected). The cardiac electrophysiological community has been sharing models first informally and now semi-­‐formally through CellML for decades and much effort has been put into the task of these translators, markup languages and tools. I think having the flexibility to run models on different machines is an important aspect of a modeling library. https://www.cellml.org/ is an example of a model repository xml code that can be converted into C, Fortran, Matlab, etc. as well as corresponding tools to run/analyze models. The CellML model repository has been curated and the markup language includes the feature of ensuring that all units are consistent in a model, which is a common translation error that is not easy to catch when debugging and combining models. Gary Mirams is developing a framework they are calling “Functional Curation” to compare and contrast models by running them through the same battery of tests. https://travis.cs.ox.ac.uk/FunctionalCuration/index.html In addition there are numerous software packages available for free to run simulations, visualize the results, create model emulators, etc., I’m most familiar with whole heart simulators and I would think having either a list or housing these significant resources would be quite useful. [We] would be happy to compile a list of relevant urls and more information for cell models and whole heart models if the information can be made public. Response / Feedback This phase of the library is focused on inclusion, so the tools, methods and other repositories you mentioned can be linked to within this phase 1 library scheme. The architecture is designed to allow any type of file and facilitate community engagement. Perhaps we will find that this infrastructure allows for a common place to discover these other underutilize resources. CHANGE SINCE LAST UPDATE At a minimum, assessment must be updated as part of each phase review. Highlight any significant changes. Changes have been updated in red and hyperlinked at the top of the document.