Posts tagged Testing
Bing Testing New Search Results Design
Feb 8th
Bing is testing a new design for the search results layout. The new design seems a lot cleaner, fresher and more organized compared to the older design. A reader tipped us off listing out the key differences: Vertical tabs removed Top and bottom search boxes widened Search button icon changed…
Please visit Search Engine Land for the full article.
View full post on Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing
Mobile Carriers and OEMs Get Android App Testing Cloud from Apkudo
Feb 8th
When developers think of application testing, it always centers around how an app will perform on a particular device. This is especially important in the Android ecosystem that has upwards of 300 devices from a variety of original equipment manufacturers worldwide. From the inverse perspective, nobody ever thinks of the testing needs of the carriers and OEMs.
Cloud-based testing platform Apkudo thought about manufacturers and carriers with a new release of device analytics platform. Manufacturers can now test devices against the top Android apps before releasing. The idea is that if a device is tested from the supplier side, fewer handsets will be returned by consumers, potentially saving manufacturers billions of dollars.
Sponsor
Apkudo tests with what it calls a “device cloud.” The configuration of more than 300 Android devices are set up in the cloud and mobile app developers can run their projects through that cloud to make sure it will work across OEMs and Android system versions.
For Apkudo’s device analytics, the opposite approach is taken. Manufacturers and network operators can test their apps against the contents of the Android Market. Apkudo will run a device against the top 200 apps in the Market to test functionality with the touchscreen, keyboard, audio, device access (accelerometer and GPS, for instance) along with performance characteristics.
This should provide developers, network operators and manufacturers with tools against Android fragmentation. As we noted last week, there is actually less fragmentation of Android devices than many think, with the optimal Android handset running on a 4.3-inch screen on version 2.3 Gingerbread. Yet, with the sheer volume of devices and applications available in the Android ecosystem, testing is still one giant headache.
Apkudo can speed up on the process that OEMs must go through to test devices. According to CEO Josh Matthews the process normally takes 6-8 weeks. Apkudo says it can do it in three days.
Device analytics will break down the results into two categories: characterization and optimization benefits. Characterization benefits help operators target competing devices while expanding their own portfolios. Imagine it as a bench mark against the rest of the ecosystem. Optimization benefits recommends how devices can be made better before release to be truly competitive in the market place.
The first U.S. carrier to sign on with Apkudo is MetroPCS. Apkudo also has agreements with “most major OEMs” in the Android ecosystem.
App developers should be happy with Apkudo’s testing abilities because it means that the OEMs could have a more efficient testing program to make sure apps work on their devices. When it comes to app functionality on Android, developers need to work the manufacturers and carriers to ensure a quality experience. The end of fragmentation, after all, is a two way street.
View full post on ReadWriteWeb
HP Aims to Redefine Apps Performance Testing with Cloud Platform
Feb 7th
When mobile users feel they don’t like how their apps perform after the first trial, some 75% of them won’t launch the app again. That’s the metric cited by engineers and marketers at HP Software, who note that this first wave of mobile apps brought forth by the iPhone has resulted in a glut of programs that make even the best performing mobile hardware into a pocket full of silicon cement.
This morning, HP begins a repurposing of the performance testing tools for Web sites that it gained through the Mercury Interactive acquisition of 2006, for the mobile apps era. It’s unveiling what it calls “LoadRunner-in-the-Cloud,” complete with hyphens. It will act as an off-premise testing platform for mobile apps that are deployed as services, simulating the activity generated by thousands of users simultaneously to gauge the resilience of servers and resources. This way, you might not have to disappoint three-fourths of them to learn how well your service holds up.
Sponsor
Better scalability through experimentation
“The application architecture itself is the performance bottleneck nine out of ten times,” says Matt Morgan, HP Software’s global senior director of solutions marketing, in an interview with RWW. “By monitoring these services and knowing how long a transaction spends waiting for a data retrieval or a logic process, or some other storage function to occur, you can pinpoint the modules inside the service that have the most potential to slow an application under load. You can find out which services are not scaling.”
When applications are deployed on PaaS platforms such as Heroku and Windows Azure, Morgan says, a great deal of the complexity of how the software interfaces with the hardware is abstracted into obscurity. The architectural concept that I dubbed composite applications in 1991 has expanded to a seemingly unmanageable number of tiers. With the shift to mobile architecture, much of the burden of providing performance has shifted off of the front-end client, and onto the server. And in-between those two tiers are any number of platforms. “So this idea that the software is a composition, gets even more complicated,” he remarks.
“We correlate the front-end story to the back-end problem. And if you think about just the complexities of performance monitoring, if you don’t do correlation, you can end up with an enormous collection of logs and metrics that don’t actually mean anything to the tester,” he continues. “The tester really cares about, how many users can the system support, and what will these users experience when they do concurrently hit their system?”
Mobile apps typically break at some point, and Morgan notes, they don’t bend very much before they do. Maybe an app performs well with 300 simultaneous users, and then fails completely at 325. So LoadRunner-in-the-Cloud (hereafter LRC) finds the breaking point, which is typically somewhere. Once that’s done, it relies upon feedback provided by a vast network of back-end monitors, probing such factors as SQL queries, server metrics, and diagnoses of the method calls being invoked inside the service. “By correlating how much time it takes for a transaction to hit these things, you can actually attain a pretty clear picture, and start to show that the areas of your application that are causing problems have a distinct performance impact.”
The protocol HP uses for emulating user activity in AJAX Web applications, called TruClient, is explained in this video. TruClient has been extended for LoadRunner-in-the-Cloud.
Identifying your app by how and where it breaks
The result is a kind of “stress footprint” that characterizes the resilience state of your mobile app. The space in which this footprint appears is the scenario, which is LRC’s term for a repeatable test. Each test helps establish a firm baseline, which is then replicated identically for different problem sets – different numbers of users. This way you’re testing how the same code scales up, including with each increment – LRC adds test users on an incremental scale, not logarithmic. “You’re trying to determine, if the exact same actions take place on the server, do things improve with the change?” says the HP senior director. “Scenarios allow you to digitize that load, creating a one-click re-execution capability, which is very important in an iterative world. You run your load test, you identify your problem, you make your change, and you go back to your scenario and run the exact same test.”
Overlaying the results gives you your best metric as to performance, which in a cloud-based scenario is indeed capable of improving with incrementally added users.
Results from previous load tests, including with earlier builds of your apps, are recorded as snapshots. New test results can be overlaid atop these earlier ones, in order to determine what code changes made the biggest impact. “We give you the capability to leverage that information inside of an operational monitoring solution, but if you wanted to monitor a Web or mobile app going forward for functionality, and you want to have visibility to the way it should run, you can use the metrics from LoadRunner to compare against what it’s actually doing in production. That gives you the snapshot of the lab world, where everything works, to the production world where everything’s real.”
LoadRunner-in-the-Cloud is being offered now to HP partners in the U.S. and Canada, and will be rolled out through OEM partners on their own timetables. Pricing will be determined by the party making the sale.
No, no, not that Lode Runner! Somebody get our graphics department off the Apple II and replace this!
View full post on ReadWriteWeb
When A/B Testing Prices, Proceed With Caution
Jan 25th
I had an interesting A/B testing experience over the holidays. This time, it wasn’t an A/B test that I was running, but rather an A/B test in which I was an (initially) unsuspecting participant. It reminded me of the negative side effects that certain kinds of tests can have on customers —…
Please visit Search Engine Land for the full article.
View full post on Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing
App Testing Catches Up With the New Era of Gesture-Based Input
Jan 23rd
Gesture-based input is the present and future of computing. We have added whole new meanings to words like swipe, pinch, zoom and flip. For mobile developers, reconciling touch-based input with design and functionality goals in apps has become a problem. Testing gestures in an app is time consuming and problematic.
A “cloud testing” company by the name of SOASTA wants to change that. It has come out with several new products today to help developers test gesture-based input for mobile applications. SOASTA said last year that 75% of all mobile and Web apps go live without ever being scale tested. By merging the cloud and new touch modules, SOASTA believes it has evolved app testing to finally catch up with input methods.
Sponsor
SOASTA is unveiling CloudTest, a tool that captures all user actions and gestures on real devices. SOASTA is attempting to create a new test method that is better than the previous generation of device emulators and optical recognition. By merging its gesture-based testing to the cloud, TouchTest can examine how interfaces react to input on real devices in the company’s network.
This can all be done without tethering, rooting or jailbreaking a device. TouchTest controls the device through software and access them through their IP address. Basically, TouchTest is Internet driven. Organizations can build a “private device cloud” with devices inside their data centers or in remote locations. That means that the number of devices needed to perform large scale testing is greatly reduced.
The notion of a device cloud is nothing new. There are other services like DeviceAnywhere, Apkudo and Perfecta Mobile that apply similar solutions. The notion is to test an application against dozens or hundreds of devices at one time by cross-examining them through device and operating system specifications in the cloud.
With TouchTest, the idea is to see a certain swipe/pinch/zoom etc. movement will work against various UI changes. What happens in an app across platforms when someone makes this gesture?
That swipe could be somebody trying to write the number “2″ or dragging an icon from one folder to another. TestTouch can record the movement and test it in a variety of circumstances.
“TouchTest captures not only the start and end points of each gesture, but the journey between and the speed with which the gesture is performed,” the company said in a press release. “Because CloudTest tests inside the app, all actions and gestures are captured at the object level creating tests that are more stable across releases, even as the user interface (UI) changes.”
Outside of testing services like DeviceAnywhere, SOASTA says that its biggest competition is actually developers testing in the old-fashioned way: fingers and eyeballs.
“Optical recognition, or screen scraping is flawed for testing, ” said Tom Lounibos, CEO of SOASTA. “It is brittle, there is no continuous framework … we saw a specific set of opportunities with TouchTest.”
CloudTest lite is available for developers right now while the fuller, paid versions for iOS and Android will be released shortly.
Is there a market for CloudTest and TouchTest? What have been your experiences with gesture-based app testing? Does the manual approach work for you or does something like CloudTest make sense? Let us know in the comments.
View full post on ReadWriteWeb
Monetate Launches Agility Suite, Enables Unlimited Real-Time Testing & Updates Without IT
Jan 16th
Monetate Agility Suite launches today, giving marketers the ability to run tests and make site changes on the fly, even from a tablet or smartphone. Five products inside the toolkit enable unlimited A/B or multivariate testing with real-time data,…
View full post on Search Engine Watch – Latest
Shim: A Node.js App for Simultaneous Web Surfing and Testing Across Devices
Jan 10th
Cross-browser testing is crucial when developing Web sites, but it can be a royal pain to have to refresh each browser across several devices to see how pages are rendered. To help cut down on the pain, there’s Shim, a Node.js app that lets you sync browsing sessions over several devices on the same Wi-Fi network. All you need is Node.js, Shim and a Mac running 10.6 or better with wired and wireless connections.
Sponsor
About Shim
Shim came from the Boston Globe Media Lab as a way of testing Boston Globe sites across different browsers. It’s written by Chris Marstall, who holds the "creative technologist" title at the Globe and was released on GitHub after requests to make it available outside the Boston Globe Media Lab.
It works by using a Mac’s Internet Sharing feature and using it as a transparent proxy for all the other machines on the network. So you need to connect all the devices you want to test to the shared Wi-Fi network, while connecting to the Internet on the Mac via Ethernet.
Traffic is redirected to the Node.js server that you’ll run on the Mac, and when you bring up a new URL on the Mac it’s sent to all the connected browsers.
Using Shim
To start using Shim, you’ll need a Mac with 10.6 or later running Node.js. Note that the Shim README points to some outdated Node.js packages on a Google sites page. I’d head over to the official Node.js site instead if you don’t have Node.js running on your Mac already.
Then grab Shim from GitHub and run the setup, configure Internet Sharing and restart the Mac. You then need to run the configure_proxy.sh script and then start Shim with node shim.js.
One Shim is running, connect to the Mac’s Wi-Fi and surf over to Google with every device you want to use Shim on. You don’t need to install anything on the client computers, so long as they have a reasonably modern browser that supports JavaScript. (So you probably won’t be able to use Shim with, say, w3m. Sorry.) When you search for "attach" in Google, Shim will redirect browsers to the URL that the main system is browsing. There’s also a bookmarklet on the Shim homepage that you can use.
Shim has a couple of other nifty features that are of interest. For example, Shim shares cookies between browsers. You should be able to log into a site and have that login persist across other sessions, assuming the site uses cookies for logins.
You can also use Shim for slideshows, so instead of testing you could use Shim to display a slideshow to multiple users. See the README for more on how to use Shim.
One note on the license. There doesn’t seem to be one, except asking that the "About Shim" section be left in the README file. If you’re using Shim, this shouldn’t be any real cause for concern. If you’re thinking of distributing Shim, though, that might make legal a bit nervous. It’d be nice if the Globe folks would pick a well-known and OSI approved license.
There’s also the small matter of requiring a Mac to use Shim, of course. Many Web developers may have Macs, but not everybody. It’d be great if someone hacked Shim to run on Linux and Windows, too.
Aside from that, Shim seems like quite a handy little project. Anybody using it for testing yet?
View full post on ReadWriteWeb
Facebook Testing Private Messages For Pages
Dec 12th
Facebook just confirmed with us that it’s testing a feature that would allow people and Pages to communicate privately. This update was first spotted by communications agency WeAreSocial.sg. The private messaging feature would give customers the opportunity to speak privately with page owners. This is actually another Twitter-like move by Facebook.
On Twitter, brands that follow their followers back can easily communicate through private DM messages. On Facebook, however, most of the conversation between brand and user takes place on the wall, something that not all customers may feel comfortable with.
Sponsor
The other positive aspect of private messages for brand pages is that every comment doesn’t get swept into the Facebook news feed for all of a user’s friends to see. For brands that are looking to promote positive notes from fans, however, a positive message sent privately doesn’t help exposure.
Not all brands have to comply with private messages, however. Page owners will be given the option for private messages.
A visitor doesn’t even have to “Like” a brand’s page in order to send a private message. Every sent message ends up in the Messages inbox along with notes from Facebook friends. The page receives a notification once the message has been sent.
WeAreSocial.sg reports that this feature is only available in Asia, suggesting that this may be an East-to-West rollout like Facebook Timeline.
View full post on ReadWriteWeb
A 4 Step Multivariate Testing Process That Works
Nov 22nd
When we talk about website testing, we mostly talk about what to test: unique selling propositions, headlines, calls-to-action, etc. It’s equally important to understand howto test (i.e., the process that produces better results).
The more…
View full post on Search Engine Watch – Latest
YouTube Testing Major, Google+-Centric Redesign
Nov 15th
YouTube is testing a new design that would clean up the site, make subscriptions more important, and integrate Google+ functionalities, The Next Web reported. The test is currently in early trials with a small number of users.
Images courtesy…
View full post on Search Engine Watch – Latest
