PUBLIC MARKS with tags performance & database

Redis is different than other database solutions in many ways: it uses memory as main storage support and disk only for persistence, the data model is pretty unique, it is single threaded and so forth. I think that another big difference is that in order to take advantage of Redis in your production environment you don't need to switch to Redis. You can just use it in order to do new things that were not possible before, or in order to fix old problems.

NewToy (now Zynga with Friends) the folks that brought you the wildly popular mobile game Words With Friends, also have served up some open source code for your enjoyment. John Nunemaker and Geoffrey Dagley have created Toy Store, an ORM that promises to let you completely change your data store in a couple lines of code.

The Drizzle project is building a database optimized for Cloud and Net applications. It is being designed for massive concurrency on modern multi-cpu/core architecture. The code is originally derived from MySQL.

A performant key-value database. A PHP module is available, that lets PHP access to Redis in a performant way.

"Jet Profiler for MySQL is real-time query performance and diagnostics tool for the MySQL database server". I love the Query ratings feature and the live query observation - should replace the weak use of MySQL proxy.

MySQLTuner is a script written in Perl that will assist you with your MySQL configuration and make recommendations for increased performance and stability. Within seconds, it will display statistics about your MySQL installation and the areas where it can be improved. It’s key to remember that MySQLTuner is a script which can assist you with your server, but it is not the solution to a badly performing MySQL server. The best performance gains come from a thorough review of the queries sent to the server, and an evaluation of the MySQL server itself. A qualified developer in your application’s programming or scripting language should be able to work with a MySQL database administrator to find improvements for your server. Once the server and application are optimized well, you may need to consider hardware upgrades to the physical server itself. This is a really useful tool for helping to optimize MySQL performance.Understanding the various my.cnf variables and how they affect performance can seem really complicated but this tool takes some of the pain away and makes it easier to understand the effects of each variable.It is especially useful to be able to see the global memory usage, memory usage per thread and the maximum possible memory usage - that is really valuable information that is otherwise complex to calculate.

Hypertable is an open source project based on published best practices and our own experience in solving large-scale data-intensive tasks. Our goal is to bring the benefits of new levels of both performance and scale to many data-driven businesses who are currently limited by previous-generation platforms. Our goal is nothing less than that Hypertable become the world’s most massively parallel high performance database platform.

Opening a database connection is a resource intensive and time consuming operation. Connection pooling increases the performance of Web applications by reusing active database connections instead of creating a new connection with every request. Connection pool manager maintains a pool of open database connections. When a new connection requests come in, the pool manager checks if the pool contains any unused connections and returns one if available. If all connections currently in the pool are busy and the maximum pool size has not been reached, the new connection is created and added to the pool. When the pool reaches its maximum size all new connection requests are being queued up until a connection in the pool becomes available or the connection attempt times out.

Spotlight® on MySQL diagnoses MySQL problems in real time by graphically displaying all database activity in an intuitive user interface, enabling you to respond quickly to issues that need attention. With Spotlight on MySQL, DBAs or developers running Windows can drill down to locate in-depth information on the source of a performance issue. And because the run-time nature of your MySQL databases is unique, Spotlight calibrates itself to a baseline of normal database activity. This allows Spotlight to automatically establish the speed of the internal flows and other activity indicators. When a threshold is met, Spotlight notifies you using a visual or audible alert. Spotlight on MySQL supports MySQL versions 5.0 or later that employ the InnoDB storage engine.

Apache Synapse offers a wide range of connectivity, including HTTP/S, JMS, (S)FTP, SOAP, WS-ReliableMessaging and WS-Security. The project has been designed to allow administrators to support advanced functionality such as load-balancing, throttling, and failover without writing code. The new Apache Synapse 1.1 release includes many enhancements based on feedback from users, including * Apache VFS based file transport - supports File System, FTP, SFTP, JAR, ZIP, TAR, GZIP, Mime * Scheduled Task support makes it simple to run repetitive tasks * XQuery mediator - simplifies XML transformation with the XQuery standard * POJO Command mediator - allows the creation of message-independent mediation logic * DB Report and DB Lookup mediators - support message augmentation and database logging * Cache and Throttle mediators/enhancements - improve performance and manage load on existing services * Split/Clone/Aggregate mediators - support batch processing of large messages * Improved logging and tracing support With asynchronous support for HTTP and HTTPS, Apache Synapse has very high scalability under load, supporting thousands of concurrent connections even with limited threads. Combined with streaming XML support, Apache Synapse has class-leading performance results. The Apache Synapse code and binaries are available from the website at http://ws.apache.org/synapse

This is a set of rules of thumb for setting up your PostgreSQL 8.0 server. A lot of the below is based on anecdotal evidence or practical scaling tests; there's a lot about database performance that we, and OSDL, are still working out. However, this should get you started. All information below is useful as of January 12, 2005 and will likely be updated later. Discussions of settings below supercede the recommendations I've made on General Bits.