Automated Testing on Smartphones and Tablets Whether your organization is looking to engage more deeply with customers, improve workforce productivity or reduce administrative costs - mobile apps and websites are becoming essential assets. As such, you must assure a quality user experience across the multitude of mobile devices, networks and platforms out there. - How do you gain adequate coverage across the fragmented device market to ensure consistent, quality service delivery of your mobile products?
- Are your mobile testing tools efficient and scalable enough? Can you achieve adequate coverage in a reasonable timeline with affordable resource levels?
- Do you have the tools necessary to effectively manage mobile quality and scale with the increasing demands of your organization's mobile initiatives?
- How do you enable remote or distributed teams that may not have easy access to devices or networks?
- How do you deliver reliable, quality mobile content while eliminating costly product issues in the field that are a result of error-prone manual testing?
Keynote DeviceAnywhere’s Test Center Enterprise Automation is a cloud-based, enterprise SaaS solution that automates the testing of all of your mobile apps and websites. It is highly scalable and purpose-built for mobile apps and websites. With TCE Automation, QA teams can quickly create and automate test scripts to capture, verify and replay real user interactions on live mobile devices. The solution combines the convenience and efficiency of test automation with the accuracy of live mobile interactions to generate reliable, reportable and actionable results. | | Key Capabilities |  | Support for All Smart Devices - Test Android, iOS, Blackberry, Windows Phone 7, WebOS, etc. – all from the same platform
- Advanced support for HTML5 web-based app development including object level scripting - so you can run one script across all devices
Easy to Use Scripting Interface - Use our multi-tiered scripting technology to -
- Create keyword driven and business process/requirements driven test scripts
- Easily create high level device-agnostic test scripts that can be used across multiple devices – making test script maintenance and porting a breeze
- Specialize specific test cases for specific devices
- Use our visual drag and drop interface or program directly into Java using our Java API.
- Test any app: web, messaging, client-server, downloadable, streaming media, short code, etc.
Immediate Identification and Reporting of App Issues and Defects. - Rerun required tests 3X faster than manually
- Reduce mobile testing man hours by 70-80%
Integration with Leading Test Tools - Powerful integration to extend your existing HP and IBM ALM (application lifecycle management) installations into mobile
- Utilize our Java API to program and execute scripts from custom test harnesses
| |  |
The Keynote DeviceAnywhere™ platform is architected for maximum implementation flexibility so you can use it the way that best fits your business needs including: - On-demand access to public devices
- Private devices, securely hosted by Keynote DeviceAnywhere or on-premise
- Local devices directly plugged into your computer
For a free trial click here. For pricing information please contact us at sales@deviceanywhere.com or 1-877-338-4230 |
 | | Support for All Smart Devices | | - Test Android, iOS, Blackberry, Windows Phone 7, WebOS, etc. – all from the same platform
- Advanced support for HTML5 web-based app development including object level scripting – so you can run one script across all devices
- Purpose-built for mobile app and website testing – since 2003
| | | | | | | | Local Plug-and-Play and Remote Online Testing | | - Your engineers and testers can plug-and-play local devices into their desktop computers for manual and automated testing
- Devices can also be set-up in an enterprise lab environment (on-premise or in the Keynote DeviceAnywhere™ enterprise cloud) for sharing across local and remote teams
| | | | | | | | Control All Aspects of Devices | | - Control device hardware functions like accelerometer, power detection, backlight detection, power on/off, slide open/close, battery connect/disconnect, SIM card, and more
| | | | | | | | Powerful Scripting Interface | | - Advanced support for HTML5 web-based app development including object level scripting - so you can run one script across all devices
- Use our visual, WYSIWYG drag and drop interface or program directly into Java
- Multi-tiered scripting enables keyword driven and business process/requirements driven test script creation
- Easily create high level device-agnostic test scripts that can be used across multiple devices - making test script maintenance and porting a breeze
- Test any app: web, messaging, client-server, native, streaming media, short code, etc
- Take advantage of optimized image-matching algorithms that can account for small changes in pixilation
- Leverage market-leading Text Recognition capability (OCR)
- Use for manual, fully-automated, and partially automated test cases
| | | | | | | | Flexible Test Execution | | - Run ad hoc or schedule for specific days or hours
- Integrate with build process; automatically kick off BAT, UAT, functional, regression tests
- Perform end-to-end integration testing
| | | | | | | | Auto Capture, Verification and Replay of Live User Interactions | | - Get immediate identification and reporting of app issues and defects
- Run in debug mode; easily pinpoint where the failure occurred
- Scheduler runs test cases at your convenience
- Test Cycles can be created to run multiple test cases across multiple devices
- Dashboard enables users to view devices in real-time as tests are executing
| | | | | | | | Get Huge Time Savings | | - Reduce mobile testing man hours by 80%.
- Run tests overnight and 24x7
- Keynote DeviceAnywhere automated tests execute 3X faster than manual testing
| | | | | | | | Web-Based Reporting Portal | | - Streamline communication between engineering and QA teams
- Publish test results to a central repository for easy sharing and reuse
- Customize reports (screens, video captures).
- Export results to your own reporting environment.
- Step by step test results are saved, including screen shots or video
| | | | | | | | Limited internal scripting resources? Keynote DeviceAnywhere Professional Services can help! | | - Script writing services, to offload overloaded internal teams and get up and running faster
- Scripting framework that internal resources can easily build on
- Additional training for your specific use cases – helps you get the most ROI out of the system
- Ongoing script maintenance and management services
| | | | | | | | Integration with Industry Leading Application Lifecycle Management (ALM) Systems and Java API | | - Extend your existing ALM installations into mobile. Fully integrated with:
- Program in Java, utilize custom test harnesses
- Access a large pool of qualified testing engineers with Java skills
|
|  |
The Keynote DeviceAnywhere Test Automation API empowers users to interact directly with Keynote DeviceAnywhere devices through a powerful Java interface.
Containing the underlying code for all functionality built into the Keynote DeviceAnywhere™ Studio UI, the API contains all the classes and methods to support individual device commands, as well as call into Keynote DeviceAnywhere Test Automation scripts.
Supported Functions and Features
- Send input to remote devices (hardware commands, such as connecting/disconnecting battery, opening/closing flip phone, pressing buttons, tap touch screens, etc.).
- Extract outputs from handsets (audio, video screens).
- Utilize Keynote DeviceAnywhere Test Automation scripting commands (wait for specific text, images).
- Access and execute actions (test scripts) defined through Keynote DeviceAnywhere Test Automation.
Keynote DeviceAnywhere Test Automation API Features and Benefits - Access and control Keynote DeviceAnywhere handsets with a standards-based programming language.
- Flexibly utilize Keynote DeviceAnywhere Test Automation features either through the API or through Keynote DeviceAnywhere Studio.
- Create test scripts directly in Java or use Keynote DeviceAnywhere Test Automation’s visual scripting language, and easily generate Java code from these scripts with the click of a button.
- Use Keynote DeviceAnywhere Studio’s built-in Java editor, or utilize your own Java IDE.
- View executing tests in real time through the Keynote DeviceAnywhere Test Automation API Dashboard.
- Utilize existing test frameworks by converting existing assets into Java or creating a bridge to other programming languages, such as PERL or PhP.
- Easily extend testing capabilities to other systems by scripting actions between Keynote DeviceAnywhere devices and external elements – for example, test instant messaging between a mobile handset and a PC.
- Optionally save all results into the Enterprise Portal reporting interface.
- Access multiple devices within one session – the API methods will check to make sure all required devices are available before connecting.
- Integrate your test schedules with other internal events – for example, automatically initiate certain regression tests as new builds are released.
|
- General |
- Getting Started
General What is Test Center Enterprise Automation? Test Center Enterprise Automation is a powerful software platform that allows you to construct simple to complicated tests, and execute them on real mobile handsets, remotely and locally. The software also includes scheduling capabilities to run individual tests or a series of tests, as well as reporting capabilities for tracking test success/failure and viewing detailed test results.How can Test Center Enterprise Automation help me?Smartphone and Tablet app testing is often very repetitive and time consuming. With Test Center Enterprise Automation, you can define a test once – and then execute it multiple times – from a simple menu selection. You can also schedule a multitude of tests to run overnight or while you are away from your desk, allowing you to repurpose your time to other projects. Furthermore, you can create tests to run across multiple devices and multiple platforms/OS’s.What Smartphone platforms are supported?All major Smartphone and Tablet platforms are supported including Android, iOS, Windows Phone 7, WebOS, etc.What can be automated?Anything you can do with a device in your hand, can be tested through Keynote DeviceAnywhere™ Test Center Enterprise Automation, this includes - native apps, website apps, video apps, social apps, financial/retail/media/entertainment/travel/healthcare apps, etc. Our powerful scripting interface supports the following testing features: - Pressing keys and tapping touch screens on smartphones and tablets
- Hardware control (e.g. disconnect battery, data cable, accelerometer control, etc.)
- Text recognition via OCR
- Advanced image recognition
- Tests with complex business logic, such as loops & branches
- Tests between multiple handsets
- Multi-tiered scripting to create keyword driven and business process/requirements driven test scripts
Can I use Test Center Enterprise Automation with my existing test tools?Our Java API enables full access to our device scripting and control capabilities, directly from a command-line interface or via your existing test harness. We also work closely with our strategic partners to offer tightly integrated mobile testing solutions; for example, our product works with HP Functional Testing (QTP), HP Quality Center (QC), and IBM Rational Quality Manager (RQM). Our solutions team is also always available to help you with your integration needs.Can Test Center Enterprise Automation support a mix of automated and manual testing in the same test run?Yes. Keynote DeviceAnywhere enables you to be productive immediately across all mobile handsets, even when you are in the process of automating tests. You can start with your manual tests, and later automate some or all testing steps. You also have the option to automate tests on all or only certain devices.If I’m not watching the test execute, how do I know if it worked?Test Automation can execute image, text, and certain sound comparisons – becoming a virtual “viewer” of the handset. Therefore, if your test involves checking to make sure a certain phrase or image shows up on the device, you simply define that phrase or image and Test Automation will let you know if it appeared or not. In addition, to account for the variety of screen sizes and resolutions, Keynote DeviceAnywhere offers a text and image matching solution that is highly adjustable to account for variations in pixel, color, position, etc.Lastly, Keynote DeviceAnywhere Test Center Enterprise Automation has the option to record and store all test results. Screen-by-screen – so you can always see exactly what went wrong in a test. Can I re-use the test written for one handset on a different handset? If yes, how much re-writing is needed? You have the option to architect your tests into two levels: device-independent business logic (Keynote DeviceAnywhere Test Cases and Test Cycles) and device-specific actions (Test Actions and States). These two levels enable you to implement test scripts that can be easily reused across multiple devices and platforms. Top
Getting Started Can I Try Test Center Enterprise Automation before buying it?Yes, please click here to sign up for a free trial.How do I create test scripts?Test Center Enterprise Automation includes an easy-to-use, visual drag and drop scripting interface. You can also create test scripts directly in Java using our Java API. Where can I learn how to write tests and view reports? To get started, you can sign up for one of our regularly scheduled webinars. Just click here!Also, our support team is also always available to answer any questions that you may have. How do I see my test results?All Keynote DeviceAnywhere Test Center Enterprise Automation test results can be viewed through the Keynote DeviceAnywhere portal. TopSupport .: Do you have live technical support? Can I get 24/7 support?We do have live, 24/7 technical support. Using e-mail to report a problem usually results in faster resolution, as our support teams are spread out all over the world. You can also call 1-877-338-4230 and select option 2 or +1650-655-6480. .: How do I report a problem?There are multiple ways to report a problem. You can click on the "Report Issue" link that can be found in the upper right-hand corner of both Keynote DeviceAnywhere Studio and the MyKeynote DeviceAnywhere Web interface. You can also send an email directly to Support@deviceanywhere.com or call support at 1-877-338-4230, option 2 or +1650-655-6480. .: What resources and training are available?In addition to our customer support representatives, there are many training resources available to Keynote DeviceAnywhere users. We host monthly webinars that demo Keynote DeviceAnywhere functionality, as well as provide documentation that will tell you everything you need to know about using Keynote DeviceAnywhere. In addition, you will have an account manager who can always be contacted if you experience any problems. .: Help! I can't seem to sign up!Occasionally, new subscribers experience problems trying to sign up from international locations. This is usually due to firewalls, pop-up blockers, and other Internet security measures. If you continue to have problems, just give us a call (1-877-338-4230, option 1), and we'll take care of it immediately. .: What if the device I want to use is taken?We often have several devices of our most popular models in Keynote DeviceAnywhere If none of the devices for the model you require are available, you can either wait for the device – in which case you will be in queue and will get notified when the device is free -- or you can make a reservation to use it at a later time; users with reservations have priority. Top
Using Keynote DeviceAnywhere .: How does it all happen? How do I connect to a device?When you log into your Keynote DeviceAnywhere account for the first time, click on the "Launch Keynote DeviceAnywhere Studio" link in the upper right-hand corner of the MyKeynote DeviceAnywhere Web pages. An automatic download of our JAVA Swing desktop client, Keynote DeviceAnywhere Studio, will then begin. Keynote DeviceAnywhere Studio is the primary interface for controlling and accessing the devices. .: How will you notify me of software updates?As our primary application -- Keynote DeviceAnywhere Studio – is a Java Swing application, a revision check is done each time it is launched in order to make sure you are working with the very latest version of our software. .: Can I make advance device reservations?Yes, we highly encourage making advance device reservations using our built-in reservation system. That way, you are guaranteed device availability at the time that is most convenient or critical for you. You can always access any available device without a reservation on a first-come-first-served basis. Please note that users with advance reservations have a higher priority than "walk-in" users. .: I've heard you mention collaboration. How do you facilitate that?We believe that mobile development is a highly collaborative process with multiple groups within an organization (QA, development, marketing) and multiple companies (developers, porting companies, consulting companies, testing companies, and publishers) working together to launch products. Keynote DeviceAnywhere provides several collaboration features to facilitate this process, including the ability to export one or multiple frames into a Web repository, export movie clips from device activity to show collaborators actual application and device behavior, and share a device session with multiple collaborators. .: What if I want a device that is dedicated to my company, so no one else can use it?We can support this. You send us the device that you want and we will build it out and set it up in a dedicated environment. Keynote DeviceAnywhere can support any device model and can deploy in any location, Contact us at 1-877-device-0, option 1, for more details on this option. Top
Pricing .: What is a “Package”? A “Package” is a set of handsets grouped together, based on a common network, manufacturer, or other characteristic. For example, devices on the Verizon network are grouped into the ‘Verizon’ package; Sony Ericsson handsets are grouped into the Sony Ericsson package. Customers can choose which packages to subscribe to, based on which networks, manufacturers, or other characteristics of handsets you want to deploy your mobile applications to. You may also create a Custom Package; see What is a Custom Package? below. Click here for a list of available packages and handsets contained in each package. .: What is a “Custom Package”?A Custom Package is one that contains only those handsets chosen by the customer. You may choose from any of the handsets within Keynote DeviceAnywhere to create your custom package. The custom package is a good option for customers that only need to test across a small number of handsets, but the handsets span multiple carriers and manufacturers. The price of the custom package increases based on the number of devices that are included in the package. 10 device types are included in the basic custom package. Device type is defined as carrier, manufacturer and model. For example, an AT&T-US BlackBerry Bold is a different device type than a T-Mobile-US BlackBerry Bold. If the device type you choose has multiple copies in Keynote DeviceAnywhere (for example, Keynote DeviceAnywhere has 4 copies of the AT&T-US BlackBerry Bold), all 4 copies will be available to you in your custom package, but will only count as one device against the total number of devices in your package. .: Can I have both standard packages and a custom package? Which is better?Yes. You can choose as many pre-defined package you would like to have on your account, and you can also choose to have a custom package on your account. Only one custom package can be created per account, but you may add as many handsets to the custom package as you would like; there is no limit to the package size. The mix of packages that will be most cost effective is dependent on your testing needs. If you do need to test a large number of handsets that are available within pre-defined packages, it will likely be more cost effective to utilize the pre-defined packages. If you have any questions about what is the most cost effective subscription strategy, our sales team would be happy to do that assessment for you; simply contact them at sales@deviceanywhere.com. .: What are “Device Hours”?Device Hours translate to usage time on handsets in the Keynote DeviceAnywhere system. You may use your Device Hours across all handsets and packages that you have subscribed to. Your monthly invoice will provide a detailed breakdown of your hourly usage with “by-package” detail. Only the time you actually engage a device (i.e. the device is acquired by you) is counted against your Device Hours, accrued in 6 minute increments (or 1/10th of an hour). Note that you must release acquired devices to stop accrual; otherwise the system will automatically release the device after 30 minutes of inactivity. .: Do I need to sign up for Device Hours? Yes, you need to subscribe for at least a minimum of 10 Device Hours. Signing up for more hours provides the advantage of lower hourly rates. You may of course use devices for any number of hours beyondyour hourly subscription; additional hours will be billed at the same hourly rate as your subscription. .: What happens if I sign up in the middle of the month?Our billing cycles run along calendar months, i.e. from the first of the month to the end of the month. If you first subscribe in the middle of a month, your package and hourly subscription fees for that first month will be prorated based on the number of days remaining in the month. .: Can I change my Device Package or Device Hours subscriptions? You may add packages at any time; the package fee for the first month will prorated based on the number of days remaining in the month. You may cancel a package subscription at any time; however, the cancellation will go into effect at the end of the current month. Note: this means that you must cancel any packages you do not want by Midnight PST on the last day of the month. You may change your monthly Device Hours up until the 10th of the month. After the 10th, any hourly subscription changes will go into effect for the next calendar month. .: If I have a Custom Package, can I change the devices in it?Each device type in the custom package is eligible for replacement after 30 days at no cost. So if you have had your Custom Package for 30 days, you can replace any or all of the handsets that you added to that package 30 days earlier, and can do so at any time (they do not need to be replaced on day 30). If you would like to add a new handset type to the custom package, but either do not wish to remove any of the existing devices or do not have any devices that have been in the package for 30 days, any devices you add to the package (beyond the first 10) will increment the cost of the package by $10/month (or other amount if in Yen, Pounds, or Euros). .: What happens to unused monthly hours? Device hours do not roll-over across calendar month boundaries; your monthly hours will expire at the end of each month. .: When will I be billed?
Your Keynote DeviceAnywhere subscription will automatically renew each month with the same packages and hours as the previous month. You will be billed by the 5th of each month for the past month’s subscriptions and additional charges (i.e. hourly usage in excess of your hourly subscription).
.: I’ve noticed that you offer the option to pay in different currencies. How does that work? Can I change the currency that I am paying in? In February 2009, we offered the ability to pay in Euros, Pounds, and Yen. The prices are set in these currencies and do not fluctuate; the listed price is the price you will pay each month regardless of exchange rates. To view the offered prices in the supported currencies, see the Keynote DeviceAnywhere product page, under Pricing. New customers must choose their currency when they first subscribe to Keynote DeviceAnywhere. Existing customers who signed up before this option was launched will have a one-time allowance to change their currency. Please contact DABilling@deviceanywhere.com if you would like to switch to a different currency. .: Are there any other charges associated with the system? Is airtime billed? Our handsets are equipped with unlimited data service plans, and airtime is not charged. However, premium service charges you incur while using the handsets, e.g. due to downloading a non-free game or ring-tone, will be passed through to you. Note: in some cases, Keynote DeviceAnywhere has agreements with service providers that allow us to offer unlimited free paid content, for testing purposes only, on certain handsets. Please check with DABilling@DeviceAnywhere.com for a current list of handsets that offer this feature. .: When is my payment due?Payment is due immediately upon receipt of your monthly bill. Please note that if payment is not received within 45 days, Keynote DeviceAnywhere may disable your account. .: Can my subscription be used by other employees of my company, or outside contractors? Yes. An unlimited number of user accounts may be created; there is no limit on users or simultaneous usage. Remember, however, that you are responsible for any charges incurred by users on your account. .: How do I unsubscribe? If at any point you want to cancel a Keynote DeviceAnywhere subscription, you can make an unsubscribe request by visiting the MyAccount tab in MyKeynote DeviceAnywhere and selecting the “Unsubscribe” button at the bottom of the page. You may also contact us directly at sales@deviceanywhere.com.
Why Automate?
The mobile testing process is not suited to a one-size-fits-all approach. You have different goals that require different types and levels of testing at different stages in the process. These needs dictate whether it makes sense to test manually or to automate your testing; or whether to use private or public devices. The key variables that need to be considered are: - the number and types of test cases
- the frequency of testing required
- the methodology needed to conduct proper testing
- the number of different devices you need to test on
The table below compares these variables and the optimal test strategy for each: Automated testing may not be right in every mobile testing scenario, but in most cases it can save significant amounts of time and money and improve the effectiveness of your QA testing process. In the two charts below, you can see the incremental savings using automated testing vs. manual testing over a three year period. While the costs for manual testing continue to rise, the costs for automated testing level off, and the ROI increases significantly. Big Benefits Automated testing can deliver a number of significant benefits for your testing organization including: Saving Money - Automate smoke tests and regression tests to reduce costs vs. manual testing
- Find bugs earlier in the process when it is cheaper to fix them
Execute More Tests in Less Time, Speed Time to Market, Expand Your Test Coverage - Execute 2.5 – 3X faster than manual tests
- Schedule tests to run offline; utilize up to 24 hours of each day
Improve App Quality - Find more bugs, increase test case coverage, reduce tester fatigue, and offload testers from repetitive tasks, allowing them to focus on quality
Expand Device Coverage - Address quality in an increasingly fragmented device market
- The more models you test against, the more efficiency you gain
When Should You Automate? Once you understand the benefits of automation, the question becomes when? Early in the development process it usually makes sense to conduct your testing manually while you refine your test strategy. Then, once you start to standardize your testing processes, automated testing starts to deliver significant benefits. Are you Ready to Automate?
To leverage the benefits of automated testing, there are a few questions you will want to consider: - Are testing resources available?
- Do these resources have test automation experience?
- Are your Smoke and Regression test cases defined?
- Is your team familiar with the variety of popular device platforms you are targeting?
If not, Keynote DeviceAnywhere™ Professional Services can help get you there quickly and easily. We can provide highly skilled resources and services including: - Script writing - to offload overloaded internal teams and get you up and running faster
- A scripting framework that internal resources can easily build on
- Additional training for your specific use cases – to help you get the most ROI out of the system
- Ongoing script maintenance and management services
To get started with a free trial of Test Center Enterprise Automation, click here or contact sales at sales@deviceanywhere.com or 1-877-338-4230. |
The following resources provide additional ways to learn more about Keynote DeviceAnywhere™, Test Center Enterprise Automation and best practices in mobile app testing and monitoring.
|
|
Contact Us 877-338-4230 - option 1 (U.S.) +44-755-401-3466 (EMEA)
|