Transition to Drupal @ SF State
Presented By: Kevin O'Brien
Division of Information Technology
http://www.nowarninglabel.com
@nowarninglabel on Twitter
nowarninglabel on drupal.org

Campus: San Francisco State
University
1600 Holloway Ave.
San Francisco, CA 94132
Choosing Drupal
Began with search for WCM for DoIT
Tried many (Plone, OpenCMS, Joomla, etc.)
Settled on Drupal for functionality and a11y
1 virtual server, no real environment
Started with 4.7x quickly moved to 5.x
Some other sites run on Drupal independently
Ancient History
Portals required on CSU campuses
•
•
Project Approach (IBM
Websphere)
–
Collaboration between SFSU Team
and IBM Team
–
Approach according to the
established IBM Method
An emerging consensus regarding
portal development includes the
following major best practices and
considerations:
–
There should be one AND ONLY ONE
horizontal portal on campus;
–
Portals should be developed
What really happened
•
Portal took 2.5 years to implement base
features. Required constant maintenance
•
Other groups/depts. refused to buy-in,
saw IBM WebSphere as a “black box”
•
Project Manager/Maintainer quit
•
Said hey “How about going to Drupal,
they have it running successfully, maybe
would take 2.5 months” on his way out the
door
IBM Websphere
•
Java Environment
•
JDK/JavaEE/JSR 168/WSRP
•
Portlet Factory
•
Monolithic
•
Broke weekly
•
Few expertise
•
No community
•
Only support is PMRs
The A Team
•
Me – Architect /
Developer / Designer
•
(~60 hours/week)
•
Supakit “superkid”
Kiatrungrit – Developer
•
(~50 hours/week)
•
Bora Kou – Systems
Admin
(~25 hours/week on
this proj)
•
Jason D'Silva –
Student Asst. Developer (10
hours/week)
•
Management position
was actually vacant
Scrum style agile development
•
What we liked:
–
Quick
turnaround
•
What we didn't
like:
–
Scope creep
–
Weekly
feedback
–
Loss of
resources
–
Clear deadlines
–
–
Daily checkins
Cramming at
end
–
Learning a lot
–
Not enough
time for
testing/QA
–
Working a lot
Demo

[demo] of SF State Drupal Portal

[demo] of DoIT WCM
Architecture
•
2 Dedicated RHEL 5 boxes run Apache (Xeon 2 ghz + 4 GB RAM)
•
2 Dedicated RHEL 5 boxes run MySQL (Xeon 2 ghz + 4 GB RAM)
•
1 Dedicated box runs MySQL cluster mgmt node (+ MMM
replication)
•
Load balancer in front + 2 mysql db connection scripts
Big IP Load Balancer
MySQL DB
connection test
script
Apache+PHP 5.2
Apache+PHP 5.2
MySQL Cluster 7.x
MySQL DB
connection test
script
MySQL Cluster 7.x
MySQL
Mgmt Node
Drupal & MySQL Cluster :(
•
•
•
•
Drupal will not run out of the box on ndbcluster tables
Problem: menu_router table and locale_sources tables
Kris Buytaert presented on the topic
A bug was filed with MySQL, rejected because
ndbcluster doesn’t support row lengths of that size
• Issues on d.o. filed to address this in core
http://drupal.org/node/391130 and
http://drupal.org/node/703916
• So how can we make this work?
Making Drupal run on MySQL
Cluster
• Split the menu_router table into smaller, more
manageable tables
• Remove the key reference on locale_sources or change
the key data type to be something other than text/blob
• Provide documentation on install for the other issues
• Full instructions are available here:
http://www.nowarninglabel.com/home/node/42
Useful Links for MySQL Cluster
• MySQL Storage Engines
• MySQL Cluster
• The NDB Storage Engine
How to set up a load-balanced MySQL Cluster
SF State Custom Modules
User Interface Modules
Utility Modules
benefitspps
deletecookies
distributiongroups
encryption
emergencycontactpps
langhide
financialaidofferletter
printer
financialaidofferletteradmin
sfsulogout
militarystatus
shibbolethdata
namechangeselfservice
validator
nextmuni
xmlservice
officialtranscriptrequest
sfsucalendar
sfsulibraryhelp
Getting the Community to
Develop


Offered two trainings
–
1) How to install Drupal on your local
–
2) How to create Drupal modules
–
3) (Future) How to create web services
Need hand-holding
How to get a Module (portlet) into Portal
First content should be discussed with the oversight group
before beginning development to ensure Portal is the
correct place for it
Some departments or groups may have a page in Portal
that they are responsible for and control the content for
All changes must go through the development cycle
Development cycle
Develop code
•Run unit tests
•Check into SVN
•
Additional development
•Interface testing
•Tag in SVN
•
Verify automated tests
•User acceptance testing
•Content is added directly here
•Management approval
•
Automated tests run
•If any tests fail, there is no push to
production
•
Automated tests
•If any tests fail, system is rolled back to
previous build
•
Prepare
Define requirements including functionality and UI
Remember you are working within the Portal template so many UI elements are
predefined
Define interfaces to backend systems
The most common interfaces are


REST/ WebServices that should follow w3 standards
LDAP (V3)
Determine where portlet will be located and who should have access
Local development
Build portlet
Run unit tests
Unit tests should include all UI components and all backend interfaces
Check into svn
Request dev build
Backend
Work with the data and application owners of the source systems to
build the required backend interfaces
Make sure you follow the defined security model if there is one, or
work with us to define one if the system you are connecting to has no
model defined
Development (DEV)
Run unit tests
Run integration tests, finalize development
Peer review all code
Tag SVN code for test
Request Test build
Code is pulled from SVN into Dev
Test/ Quality Assurance (TEST)
Define test plan and execute all functional tests
Tests must include all functions, UI components, and backend interfaces
Perform user acceptance testing
Perform load/performance testing
Submit automated test suite for Staging and Prod
This test suite must be comprehensive
Make any content only changes directly in Test
A manager must sign off indicating the following:
He/she is aware of all changes going into production and approves them for the
specified date
The automated tests sufficiently cover all functionality
All code has been peer reviewed
The load/performance tests prove that the application meets the minimum requirements
Tag code in SVN for roll out
Code is automatically pulled from TEST at 1am and deployed into staging
Staging
Automated test suite is run
If any tests fail, production roll out is not performed
If all tests pass, promotion to Prod occurs automatically at 6am
Production (PROD)
Automated tests are run
If any tests fail, production is rolled back to previous release
Accessibility

508 compliant

All pages were run through manual checks

Theme is color contrast compliant
Security

IP based Access for back-end services

Everything is behind a firewall

Everything is behind Shibboleth auth

Core PHP Module is removed

Role based access to pages

Daily log checks (in future, archive logs in
central change mgmt repository)

Grendel-Scan

OWASP standards
Scalability
•
Handled ~350 concurrent users without
problem
•
4,600 visitors/day
•
101,000 visits and 574,926 page views
Shibboleth single-sign on & Google Analy.
•
Servers are way underutilized, use about
15% on norm
•
Load balance / Auto failover (actually
tested)
•
No outages yet over 3 mos.
Performance
•
Initial Build took 7.5+ sec. Load times
•
Used Xdebug to find drupal_http_request
was problem. Used cron to cache
•
Down to 3.5 sec load times.
•
Installed APC via PECL (quite easy)
•
(Make sure to give it enough memory)
•
Down to 1.5 sec load times.
•
Ensure Oracle db connects quit if can't
reach
Where we are Now vs. Where we
want to be



Now campus websites
are fragmented,
outdated

Not all sites use Univ.
Template, and some
don't use it correctly or
update it timely

We have a monthly
meeting Drupal User
Group

Depts. want to put
sites in centrally
managed highly
available Drupal install
for easily keeping sites
up to date
1 Drupal theme means
a consistently updated
look
Need installation
profile for point & click
Drupal is not the solution for
everything (though I love it)


Drupal is not
–
our document management system
(though it could be someday) We use
Sharepoint
–
our project management system (again
Sharepoint)
–
our help desk ticketing system (remedy)
It can be made to do all these things, but
other products are out there
The Importance of Community

Drupal.org

Bay Area / Berkeley / South Bay DUGs

IRC: #drupal-support or #drupal

On campus: http://drupal.sfsu.edu/drupal

Monthly Drupal meetups

California Higher Education Group:
http://groups.drupal.org/california-highered
Questions?

Contact: kobrien@sfsu.edu

@nowarninglabel on twitter

http://www.nowarninglabel.com