We have just released a public beta of our new database profiling tool called dataProfiler. This new tool encompasses data interrogation concepts that we have derived in performance testing and database tuning of large scale applications. Here are some of the main features of dataProfiler:
- Generate profiles for Microsoft SQL Server 2000 and 2005
- Quickly find the top 10 worse offending SQL batches in your system
- CruiseControl.NET Server Integration
- Developer bookmarks for sniffing out trouble spots in your application
- Dynamic filtering
DataProfiler is used by development teams to identity poor performing queries and help identify bottlenecks in your application. Using the dataProfiler WebConnector, you can also quickly find repeated queries within your ASP.NET application. DataProfiler interacts with SQL Server using profiling and tracing techniques to provide an easy to read specialized view of the data that can be used to quickly take control and identify pain points within your data access layer. DataProfiler interacts with SQL Server using profiling and tracing techniques to provide an easy to read specialized view of the data that can be used to quickly take control and identify pain points within your data access layer.
Download the beta today!
Do you stand out in your organization? Are you excellent at what you do? Are you a team leader? We are hiring yet again and are looking for the very best!
Join our team of professional senior level developers as we continue to grow and learn new ways to implement TDD and other Agile methodologies into enterprise level software.
We are currently looking for a Senior .Net / C# Architect / Developer to work as part of an Agile team in downtown Leesburg, VA. In addition to the requirements below, candidates must be excited about test driven development and work well in a paired programming environment.
Requirements:
• Must be US Citizen and "clearable" for obtaining Secret clearance in the future.
• At least 6+ years experience with full life cycle development in a Software Engineering capacity.
• Demonstrated experience with .NET/C# for 4+ years designing and implementing large, complex, n-tiered applications in a production environment.
• Detailed understanding of the 1.1 / 2.0 / 3.5 Common Language Runtime
• At least 2 to 4+ years experience with ASP.NET.
• At least 1+ years experience in the agile development methodology. TDD and Paired Programming a plus.
• At least 4+ years of in-depth SQL Server development experience, including: Programming with T-SQL to create stored procedures, triggers, user-defined functions, and complex SQL statements. Data Modeling, Design and Implementation, and Optimization.
• Strong experience consulting with clients to define needs or problems.
• Active involvement in the design and implementation of software systems.
• Ability to generate technical documentation
• Strong Consultant oriented skills and responsibilities, including: Excellent interpersonal, communication, and writing skills.
• .Net User Community involvement a plus.
If you are excited about test-driven development, visit http://recruiting.ent0.com to post your resume today!
Since the iPhone has what I consider a flaw in it's IMAP implemenation, we built this handy windows service that will wipe away deleted emails from your Microsoft Exchange other other IMAP based mail servers. Now you can use your iPhone and love it too! ;-)
http://blogs.ent0.com/files/folders/other_software/entry236.aspx
With the recent switch to use the Intel processor, Steven and I decided it was time to go buy the MacBook Pro as our primary laptops. I am very happy so far. I love what the Mac OS has to offer in terms of UI and how easy it is to get "gadgets" to work.
The first splurge was to buy an AirPort Express mainly for the AirTunes feature. Now I can listen to my iTunes library from my home stereo with out wires. It's friggen sweet!
One of the first utilities you wind up with when playing in the Mac world is a program called QuickSilver. It is an application launcher with some very useful macro capabilities. Since we are still programming in the Windows world, I decided that QuickSilver is a must. Within a few moments of searching I came across a windows based program called Colibri. It is not quite as robust as QuickSilver, but it works great. A must for all those who love to use the keyboard to navigate their way around the desktop.
This post is for those of you who don't already know of this great font pack from Microsoft. The font is called Consolas and it was built specifically for use in programming environments. This pack will automatically install the font pack and set it as the default font for Visual Studio 2005.
http://www.microsoft.com/downloads/details.aspx?familyid=22e69ae4-7e40-4807-8a86-b3d36fab68d3&displaylang=en
I built the Miles Ahead Farm Website a few years ago. I built it for a friend of ours with the intention that my wife would manage the content. I built some administrative functionality using the FreeTextBox control as a rich content editor. It worked out great as it allowed Amy to very easily make textual and formatting changes without much hassle. The problem was that any time she wanted to upload images (which was often), I would still have to format, upload and insert the image. It wasn't a painful process, but I'm always looking for ways to make my life a little easier.
I then came across Windows Live Writer.
"Windows Live Writer is a desktop application that makes it easier to compose compelling blog posts using Windows Live Spaces or your current blog service. "
It has a few very compelling features including WYSIWYG Authoring, Photo Publishing and a Writer SDK. Now if only I could figure out how to make Writer communicate with my dynamic website engine.
I found that Live Writer supports the MetaWeblog API.
"The MetaWeblog API (MWA) is a programming interface that allows external programs to get and set the text and attributes of weblog posts. It builds on the popular XML-RPC communication protocol, with implementations available in many popular programming environments."
After only a few moments of "using the Google", I across a blog by George Trifonov that explained and provided example C# source of how to implement a blog engine that can communicate via the MetaWeblog API.
It was quickly obvious to me that it would take less code and make more sense to write a new dynamic website engine that reads the website content from the rss formatted file that George's code was already generating.
Using the Global_BeginRequest method in the Global Application Class, I wrote a quick function that inspects the incoming request file path and rewrites the path for server processing. This means that the incoming request of "Facilities.aspx" would be forwarded to page on the server called "ShowPage.aspx" with the url in the query string. i.e. (ShowPage.aspx?url=Facilities.aspx)
protected void Global_BeginRequest(object sender, EventArgs e)
{
string filename = Path.GetFileName(HttpContext.Current.Request.FilePath).ToLower();
switch(filename)
{
case "showpage.aspx":
case "webresource.axd":
case "metablogapi.ashx":
break;
default:
HttpContext.Current.RewritePath("showblog.aspx?url=" + filename);
break;
}
} |
You will notice that I listed ShowPage.aspx and other static files as pages that will be ignored by this function and allow ASP.NET to handle the request as it normally would.
The next step is to build the ShowPage.aspx page that will handle the request for the dynamic page, query the storage file and output the website content.
protected void Page_Load(object sender, EventArgs e)
{
string linkId = Server.UrlDecode(Request["url"]);
XmlDocument doc = new XmlDocument();
doc.Load(StorageFile);
XmlNode node = doc.SelectSingleNode("rss/channel/item[link ='" + linkId + "']");
if (node != null)
{
string blogContent = node.SelectSingleNode("description").InnerText;
ContentZone.Controls.AddAt(0, new LiteralControl(blogContent));
}
} |
The code in the Page_Load method firstly loads the RSS content into an XmlDocument object and performs a search for an item with a link that matches the incoming requested url. If found, we simply grab a reference to the description node as this is where the html content is stored. In my specific implementation for Miles Ahead Farm I've added support for output caching, html page title and also support for loading web user controls for needs such as a contact form.
You can download the Dynamic Website Engine Source Code from our downloads page.
So you have a database with real data that you want to use in your testing framework and you are looking for a way to get that data out so that it can be maintained in a script and be run as part of your unit tests.
Thycotic Software has a great tool for generating a setup script for your test data.
"Thycotic.DatabaseScripter is a simple utility that generates a stored procedure called "SETUP" which will remove all data from your database and then reinsert it in the correct order. It uses the system tables in the database to identify foreign key dependencies and make sure data is manipulated in the appropriate manner."
I will start the series with the DELTA_TableExists function that is provided in the deltaHelper script. We have implemented certain guidelines on our team for writing deltas. One guideline required that all delta be repeatable. As you may have already noticed, deltaRunner enforces this policy by executing deltas in development mode two times. Writing deltas in this way force us to think about the database prior to just making changes. We use this function in our delta files to ensure that we are only adding a table to the database once. To make changes to a table, we will use a procedure called DELTA_ColumnAdder or one of a few other variants. More on this in a later post.
The DELTA_TableExists function provides an easy way to determine if a table already exists in the database. Lets say for any number of reasons you wanted to know if a table exists in your database. In this example we will do some work if we know a specified table already exists in the database.
if(dbo.DELTA_TableExists('TABLENAME') = 1)
begin
-- TABLE EXISTS ... DO SOME WORK
end
go
Or you might want to check to see the table does not exists.
if(dbo.DELTA_TableExists('TABLENAME') = 0)
begin
-- TABLE DOES NOT EXIST ... DO SOME WORK
end
go
How many times do you have to write a query against the information_schema.tables view before you finally remember the name of it and where that underscore goes?
deltaHelpers are a collection of stored procedures and sql functions that are a direct result of database refactoring. Prior to these functions the code was sprinkled all over my deltas and are now in a single place to remove duplication. With functions like DELTA_ColumnAdder and DELTA_TableExists, these deltaHelpers allow you to write better functioning deltas with fewer lines of code.
I will document the usage of each of these functions in the coming days/weeks in future blog posts. Most of the deltas are pretty well documented and have sample usages and remarks in the downloadable sql script.
First of all, I would like to thank Roy Osherove for his recent blog about dataFresh. Right away we had more exposure for dataFresh than we have ever had! For those who don't know, Roy contributes to the agile/TDD world on his blog and has earned himself the ear of many loyal readers. I encourage you to visit his blog and join in on the TDD bandwagon. Roy -- Send me some feedback once you get a chance to actually try dataFresh.
dataFresh is extremely easy to implement into your test framework. I will demonstrate just how easy it is by blogging a very simple implemenation using a test fixture base class from which all test fixtures should derive.
[SetUp]
public virtual void Setup()
{
InitializeDatabase();
}
public static void InitializeDatabase()
{
if (DataFreshInstance.HasDatabaseBeenModified())
{
DataFreshInstance.RefreshTheDatabase();
}
}
public static SqlDataFresh DataFreshInstance
{
get
{
if (dataFresh == null)
{
try
{
dataFresh = new SqlDataFresh(SampleDatabaseConnectionString, true);
}
catch (SqlDataFreshException)
{
dataFresh.PrepareDatabaseforDataFresh(false);
}
}
return dataFresh;
}
}
This example assumes that you have already generated the snapshot and that you are not monitoring delta files via your test framework. If you are interested in a solution that does just that please see the downloads section for some sample code on integrating both deltaRunner and dataFresh into your test framework.
First of all we have over 3,000 automated NUnit tests in our application! Most of these tests are in fact writting to the database. Up until dataFresh, we used a stored procedure that was coded with all our setup or test data. This procedure would be called before each test run if the database had been written to in a prior test. The major problem is that it took upwards of 4 seconds to execute this stored procedure. Thats 4 seconds for each test that writes to the database!
2(seconds) * 1500(database write tests) = 3,000 (seconds) or (50 Minutes)
So lets see, we've wasted 50 minutes of our time sitting and waiting for the database to refresh. Plus the time required to run the actual test! We quickly added a team policy that states that we only have to run readonly tests prior to checking in and let the server build and test the rest of them. Practice shows that this sometimes allows developers to "feel confident" with their code and not run any tests at all. The boottom line is that we have wasted entirely too much time and have got to do something about it.
dataFresh to the Rescue
It was pretty easy to implement dataFresh into the solution as it was already being used to initially populate the database from a baseline attempt. This particular project uses an audit database, so I setup a second instance of dataFresh to also restore this second database. There were a few modifications to the test fixture base classes to bascially remove the older stored procedure calls, and then we were ready to go.
On the initial run of our business layer unit tests we went from 30 minutes to just over 20 minutes. We realized a more than 30% performance increase! While watching the tests run, we noticed that there were a few tests that were taking a long time to execute. We decided to spend a few minutes looking at these tests and realized that we were doing some bad things in these tests that took the system a while to respond. After a few minutes of test optimizations, and another full test run of the business layer unit tests we are running to green in under 10 minutes!