Testing Tools in Alfresco
About me
• Fran Álvarez
• Zaizi Iberia Director
• Keywords: ECM, Information retrieval, testing,
semantic technologies
Twitter: @fran7alvarez
Email: falvarez@zaizi.com
Skype: fjalvarezb
WARNING
This presentation may contain technical content
Manager discretion is advised
Index
• Testing concept
• Zaizi testing stack
• Challenges
• Ecosystem
• Continous Testing
• Advice
• Evolution
• TDD in Alfresco
• Conclusions
What is ‘Test’
A test is an assessment intended to measure a testtaker’s knowledge, skill, aptitude, physical fitness or
classification in many other topics.
Test in Software
Software testing is an investigation conducted to
provide stakeholders with information about the
quality of the product or service under test.
Our measurement (aka Metrics)
• How well our code works
• How ‘well coded’ our code is
• How good our tests are
• How well the requirements are met
• How well our features responds to massively
requests
•
Our assessment
• Different ways to measure:
•
Border Line
•
100%-0%
• Acceptance criteria definition
• Example:
• 100% UT, 95% FT, 70% CC
Our test-taker…
…knowledge
The importance of testing
• Testing means:
•
In architecture:
•
•
The building where I live is not going to fall down
In aeronautics:
•
•
The plane where I’m traveling is not going to fall down
In food industries:
•
The food you are having is not going to cause you pain
Have I convinced you yet?
The difficulty of testing
• Other tested subjects know their ‘environment’
The difficulty of testing (II)
• In Software Engineering we may have:
•
Code run in different servers
•
•
•
Race conditions
Performance issues
Code is integrated with 3rd parties
•
•
•
3rd parties changed
3rd parties might not be available
Requirements change after placed in production
…
The difficulty of testing (III)
• It was never easy to properly test
Don’t waste your time testing!
We need to get more
profit from the project
Scenario
Scenario
• Lots of features
• Long duration project
• Several components:
•
•
•
•
•
•
API
Behaviours
Out-of-the-box customizations
Huge permission complexity
3rd party integrations
Etc
The ZAIZI Testing Ecosystem
Coding does not end within the code
The ZAIZI Testing Ecosystem
The ZAIZI Testing Ecosystem
• Perform unit tests for each piece of code
• Mock 3rd parties or subsystems
• Create reports for code covering and code health
• Create harnesses for APIs
• Mimic user operations for functional tests
• Create report for multiuser and multithread testing
• And more…
The ZAIZI Testing Ecosystem : Maven
• The foundations
• Full lifecycle management
• Deliver Alfresco code in any format
•
War, AMP, Jar, RPM
• Integrate some useful plugins
•
•
•
Surefire
Cobertura
Sonar
The ZAIZI Testing Ecosystem : jUnit
• Unit tests
• Maven surefire integration
•
Virtual Alfresco context
•
Clean run every time
•
Database wiping
• Does not generate the package if some test fails
The ZAIZI Testing Ecosystem : Mockito
• Mimic components, 3rd parties
• Very useful to isolate your code
•
Just test what’s yours
The ZAIZI Testing Ecosystem : Sonar
• Open Source code quality platform
• We used for:
•
Check how complex the code is
•
•
Cyclomatic complexity
Check the style of the code
•
•
Variable/Methods names
Code coverage
•
An alternative for cobertura
The ZAIZI Testing Ecosystem : Cobertura (II)
• Line coverage acceptable
• Branch covering not enough
The ZAIZI Testing Ecosystem :JMeter
• Central axis
• Running services automatically
•
HTTP: Webscripts
•
SMTP: Emails
•
Etc
• Used for creating test suites:
•
API harnesses
•
Emulate users
•
Massive requests
The ZAIZI Testing Ecosystem : JMeter (III)
• Test harness purposes:
•
Catalog of operations
•
Practical examples
•
•
Inputs, outputs
Check unexpected changes visually
DEMO
The ZAIZI Testing Ecosystem : JMeter (IV)
• Functional test purposes
•
Collecting use cases
•
Emulating them
•
Checking the response is the one expected
The ZAIZI Testing Ecosystem : JMeter (V)
• Performance test purposes
•
Basic operations executed several times
•
•
Several threads, several users
Reports for diagnosis
•
Combined with Appdynamics
The ZAIZI Testing Ecosystem : Confluence
• Used for uploading reports
The ZAIZI Testing Ecosystem : Confluence
• Checks every single jMeter operation
The ZAIZI Testing Ecosystem : AppDynamics
• Used for checking and montoring the performance
of the applications in the environments
• Detect slow transactions, memory leaks, etc
• Huge amount of options and configurations
•
Could be talking about it for hours
The ZAIZI Testing Ecosystem : AppDynamics
Automated Continous Testing
• Testing without human involvement
•
Dedicated server for testing purpose
• Continous deployment purpose
• Configurations close to production
• Repeatable, robust, auditable
Again, thanks to the customer!!
Automated Continous Testing
• Not anymore!!
Automated Continous Testing - Algorithm
Don’t become a test-addict
• Don’t ‘all-in’ if you don’t need it
•
Meet your requirements, do your testing
Leverage your code with testing, not the opposite!
Evolutioning the ecosystem
• Making functional tests more functional
• Detecting some behaviors within the log files
Ecosystem 2.0: Splunk
• Monitor Alfresco logs with ‘live’ tests
•
•
Detect unexpected outputs
Alert unexpected behaviour
“Would you like to know more?”
Ecosystem 2.0: Selenium
• Limitations with JMeter in Share
• Asynchronous calls are not catched properly
• Selenium is perfect to emulate user&browser
interaction
TDD in Alfresco
• Test Driven Development
•
Simple software development technique
•
Short development cycles
•
Focused on deliver what is required, nothing else
TDD in Alfresco
TDD in Alfresco
• Possible thanks to Alfresco and Maven-Surefire
integration
• Virtual Database and Alfresco context
TDD in Alfresco
• Benefits
•
•
•
•
•
You focus on delivering, not coding for coding
Coding behaviors, not just code
Tests are really part of documentation
Debug actual code from tests
Usually less number of defects
• Inconvenients
•
•
Tests are slow – Alfresco context starting up
Not prepared to test UI
Cost Savings
•Time saving
•
•
•
•
•
Validation could take more than 3 hours
Create, check, delete…
Project duration might be 2+ years
Do you need a calculator?
Reducing development cycle
•Moral savings
•
•
Testing the same everyday is frustrating
You don’t test the very same twice
•Cost effective testing
•
You only test when something went wrong
Conclusions
Conclusions
• Testing is a good QA indicator
• Not conform yourself with Unit tests
• Build a stack to fully test your apps
•
You will save time and frustations
• Don’t become crazy by testing
Credits (or thanks, thanks, thanks)
• Trafigura
•
Michael Phan – Technical Architect
•
James Bennett – Head of QA
•
Ian Carr – Project Manager
Thank you!