Moving on...

So I think this may be a move to the big leagues so I supposed I'm going to have to provide some real content. Fortunately, my new employer is doing lots of really cool things so I expect I'll measure up to the task.. eventually.

So my new blog is at http://weblogs.asp.net/jimjackson/. That's right! I'm a Microsoft Sponsored Blogger. Not really but it sounds cool right? Me, moveon.org and Rush Limbaugh are going to really have an impact on this election! Wait, that's not right. I'm going to post the tips and tricks that I learn while spinning up on Sharepoing 2007 and Visual Studio 2008. That's pretty much it.

JJ

Anyone can break a shovel...

I was about 12 and my grandparents' turn of the century farm house had shifted on its wooden beam foundation. The whole family was involved in the process of digging up the ends of the beams, jacking them up and laying concrete beneath them to relevel the house. Over lunch, while everyone else went inside, I stayed and continued to dig. The ground in Northern Michigan can be very rocky and at one point I struck a large stone. After about 20 minutes of excavating the stone, I thought it was ready so I wedged the shovel in on one side and attempted to pop it out. Turns out, I had only excavated the top 30% of the stone. With all my weight on the shovel, the handle promptly splintered.


I waltzed into the house proudly displaying the broken shovel as proof of how hard I was working. My grandpa looked at the shovel, scowled and yelled "Hell boy, ANYONE can break a shovel!" I was crestfallen. The proof of my hard work turned out to be nothing more than a measure of how little I knew about using my tools.


Jump forward 25 years and I find the lesson still applies. I am always impressed at how easily some developers and managers in the software provisioning industry are always trying to:

1. Use a piece of technology to solve a problem in direct conflict with that technology's expressed purpose.


2. Decide not to use a technology that is specifically designed to solve a problem because it was 'not invented here' or because it would mean learning something new.

As software developers, it is our job to enhance the state of the art in what we build. We must know enough about the problem and our tools to know what is possible, what isn't and where the lines become fuzzy. For example, a developer who will not use generic collections but insists on manually boxing and unboxing collection contents has not learned the capability of his tool set. Similarly, a developer who insists that databases are 'too much overhead' and decides to wrap a complete custom transactional solution around XML files has gone around the bend only to find an unfortunate and painful hole in his own foot. (The second example is anecdotal!)

We must, as an industry, overcome the stigma of budget and schedule overruns resulting in unusable software. To do that, we must first consider what kind of tool will best suit the job at hand and then strive to use that tool appropriately. If your software is needlessly complex and unmaintainable, it may be that you've done nothing more than break your shovel.

Web 2.0 Refreshments

Just read a blog by Sahil Malik. Smart dude and I think he gets it. Check it out: http://blah.winsmarts.com/2007-1-2007_Prediction.aspx

Team Foundation Server Links

Over my holiday, I read a book on Team Foundation Server and, true to the current low standards for industrial technology publication, the authors knew their business but not how to present it. I found the book sorely in need of an editor who spoke english as a first language. The material was trivial more than half the time and casual the rest of the time. I felt as though I were reading a TechEd transcript. Not exactly a fluent method for getting your point across in print...

At any rate, the authors did do a good job of researching related information freely available on the web. Unfortunately, it was printed and there was no ebook published with the print edition. The result is that I bookmarked all the links that I thought would be helpful and then located their actual URL (many published values are post-MSDN2). Here are the pieces I found useful.

Continuous Integration
- Continuous Integration for TFS
- MSDN Article on Using Team Foundation Build

MS Build
- GUI for MSBuild Sample
- MS Build Sidekick
- Team Build Ticker
- Visual Studio 2005 Extensibility Center
- MSBuild Community
- MS Services Build Framework (SBF) Tasks

ASP.Net Applications in Team Build
- Team System Team Build TechNotes for ASP.Net Deployment Projects
- Building a default ASP.Net project in Team Build
- VS 2005 Web Deployment Projects Information

MSF Process Integration and Customization
- The Origin of Personas
- Why there are 5 missing CMMI process areas
- Starting Point for Customizing MSF Process Guidance

MS Threat Analysis and Modeling Tool
- MS Application Thread Modeling Blog

MSF Document for SCAMPI Requirements Roadmap (CMMI)
- MSF CMMI Reference.xls

MS Visual Studio Developer Center
- MS Visual Studio Developer Center (requires free membership)

Work Item - Process Template Editor
- VSTS Customization Toolkit
- Team Plain for Team System (TFS Web Access)

TFS Version Control
- Branch Merging Primer

Team System Requirements Authoring Starter Kit
- RASK Starter
- RASK Code Definition
- RASK Download

Entity Modelling 101

This post came by the MSDN architecture forum and I thought it was a good 20 second thought for the day...

I have a really tough data modeling problem, and I'm dumbfounded. I hope there's somebody with an idea.

The problem really deals with aquatic habitat and vegetation, but I'll frame it in a more familiar context.

It's a very common scenario. An employee works for a company. The employee is assigned to a project.

Company --> Project --> Client
:               :
:               :

Employee --> Consultant

If you add this "consultant" relation between employee and project, isn't that a circular reference? Doesn't that violate like 4th normal form or something?

Is there anyone with any ideas?

RESPONSE

There are a bunch of ways to go on this. Are we talking about database design or object model? It doesn't really matter in the abstract I guess...

You've defined your entities just fine from what I can see but the magic isn't in the entities. The real work happens in the relationships between entities and I think that's what you may be missing. The best description I can give is the classic: "Is-A v/s Has-A". Here goes my 101 explanation:

Company - HAS A - Project
Project - HAS A - Client
Company - HAS A - Employee (semantics aside...)
Consultant - HAS A - Project
Employee - IS A - Consultant

HAS A implies an ownership or property relationship. So the Company object would have a Project object property or Projects collection. Project would then have a Client object property etc.

IS A implies an inheritance relationship. So a Consultant object would inherit from the Employee object because an Employee could end up fulfilling the role of Consultant. Vice President would also inherit from Employee for the same reason but the base Employee functions would always be available. Also, does a Consultant have a Project or does a Project have a Consultant? Perhaps there is a context that you must come in with depending upon the situation. e.g. I am a Consultant and I have many Projects but each Project has one or more Consultants. Depending upon which direction you are viewing from, you could have a Project with a collection of Consultants or a Consultant with a collection of Projects. Both are correct depending upon what question you are planning to ask of the model.

The language is important here because you may decide that Client - HAS A - Project and this would dramatically change the way you created your model.

Labs for Windows Vista (.Net 3.0)

The architect group I am now attending expressed an interest in the Windows Vista Developer Center so I thought I'd post it here.

http://msdn.microsoft.com/windowsvista/

When you get there, you can start digging into the labs that are available in the downloads section for Presentation Foundation and Workflow Foundation. Have fun!

ADO.Net vNext

I tried as best I could to keep the technology out of this blog and stay with espousing my architectural philosophy but, after my trip to VSLive in New York, I feel I've got to throw out my opinions. I do this because of my brief discussions with Rocky Lhotka and Pablo Castro.

I was nearly knocked out of my socks by Pablo's discussion of the Entity Framework. I finally feel validated with my current framework although my soon-to-be-published book will probably need some revisions. Ok, MANY revisions. That's not a bad thing since I haven't started writing it yet. Ah well.

So, from what I know, here's the deal with the Entity Framework. (I've got a lot more to read to see what the real deal is, but for now, I think I get it.)

You can do lots of cool things with business objects but your usefulness wanes to about 1% when they don't work with business data. Traditionally, you write a stored procedure, pull the data, put it in properties in your object, mung it, send it back, write more stored procedures to perform dml operations. When something changes, you have to touch everything. In the words of my 3-year-old daughter, that's icky.

Enter the Entity Framework, whose goal is to DEFINE THE DATA you need to manipulate, not to define HOW TO GET AND SET THE DATA. In my architecture, a logical table is a portion of an entity's data. It can be more or less than a table and it can be more or less than a record in that data. Once defined, the framework has a cache of its own decision tree specific to that entity. You can do anything you want to that data and it will find its way back into the store, business rules approving, of course.

In talking with Pablo, the Entity Framework currently generates all of its SQL on the fly. We tried that too. Its good for snap and sizzle. Not so much when 500 users ping your customer list at one time. Our answer is to build it when you need it, validate it internally (in another thread) and cache it until its no longer valid. For my system, this involves a listener that checks for schema changes (Hello SQL 2005 schema triggers!!!) and reevaluates logical tables that have become suspect.

Even at this level though, the Microsoft team is going to have performance issues. There are indeed two more facets of this problem that seems to be unaddressed. I'm interested in whether or not anyone else out there sees it.

Jim

Architectural Maxim #2

If the intelligence required to work with an architecture is greater than the intelligence required to create it, that architecture has failed.

The Architect's Job

In my company, we have Solution Architects and Technical Architects. Just about every software development methodology out there defines an architectural role in the software process. Industrially, the title Chief Software Architect or Chief Architect are in vogue. There are probably another dozen or so job titles involving the word. If you look at the supposed community definitions of Architecture at SEI, a thinking man will probably laugh till his coffee comes out his nose.

So since this is a pretty new blog, I'm looking to generate a little discussion (and traffic) on what you think an Architect's role is. Aside from attempting to sound like the smartest feller on the project, what do we do? Specifically, I want to know:

• What is the Architect's role in project management?
• What is the Architect's role in personnel management?
• Should an Architect be a trainer too?
• What is the significance of professional certifications?
• Is a degree important and, if so, what degree?
• How much time should an Architect be expected to spend off-the-job studying and researching?
• Should an Architect be an uber-geek, with knowledge of all the latest and greatest stuff?
• What role do patterns play in the Architect's day-to-day job?
• What does the Architect do in the sales phase?
• What does the Architect do in the requirements phase?
• What does the Architect do in the design phase?
• What does the Architect do in the development phase?
• What does the Architect do in the quality assurance phase?
• What does the Architect do in the deployment phase?
• What does the Architect do in the maintenance phase?

I ask only one thing of you, dear reader; do not use the words "it depends" because there is nothing you can say to me that would convince me that it depends. If you're so sure that it depends, then there are options and scenarios that you are implying but being too lazy to elucidate. Define your perspective of the options and define which are best given a scenario that you specify. Here in the real world, you have options as soon as you wake up till you fall asleep at night. Pick one. Go with it.

JJ

Architectural Maxim #1

A good architecture will help standardize project life cycles and management phases.

Benefits of the Team Architectural Approach

Run correctly, what does a good software architecture give a development team? For that matter, what does it give to the entire organization surrounding the souls trying to develop software? It is certainly expensive and time consuming. There are no guarantees that ROI will be enough to justify it. Then, of course, a good manager can't help but notice that every developer in his charge has a different definition of architecture and, for some, no definition at all.

Let me take you into my world and show you, dear reader, what it means to have an architecture. That is to say, a software application architecture that is associated with a framework based on simple development patterns. (Design patterns are fine for IT organizations that wish to justify their own existence. Development patterns are for getting the heavy lifting done.)

My Team:
Chuck: Architect
Richard: Business Analyst
Myles: Senior Developer
Sharon: Developer
Mike: Developer
Noah: Junior Developer
Kelley: Quality Assurance and Testing

There are others but they mostly just count the beans and bullets.

Richard has completed his first draft of the business and functional requirements. These are passed off to Chuck who goes into a cave to read them unimpeded. When he returns, he has lots of questions, mostly about implementation of specific requirements or the delta of one part of the requirements to another. These questions answered, he tracks down Myles for some design session madness. By the way, if Richard is worth is salt, he has provided Chuck with some prototypes as well to show off what the client expects the application to look like. Chuck and Myles go to town with:

1. Their existing architectural patterns
2. The overall understanding of the system parameters
3. The prototypes
4. A listing of the most difficult (or complex) challenges the application must address

Step 1 is determining the patterns. Chuck and Myles have patterns for listing data, searching data, editing single record entities, editing lists, navigation control, business rule validation, state management, security and a short stack of other things. Each of these development patterns has its inputs, outputs, event model and templates defined so they know exactly what each is used for and, more importantly, what each is NOT used for.

Chuck and Myles go through the requirements and categorize each screen or section of the application into the development patterns it will use. Anything that does not fit an existing pattern is the 20 side of 80/20 and is set aside for independent address later on.

Step 2 is the decision-making process to decide what to do with the 20% non-pattern entities. Each is evaluated to determine if it is truly a one-off or if the solution may generally happen again in another system. If the former, the item is assigned to Myles for development. If the latter, Myles and Chuck perform an independent design session to determine a new pattern to be added to the list. Myles codes the piece according to the pattern and his work becomes version 1 of the template for that pattern.

Step 3 is the review with Richard, Chuck and Myles. In this session, Myles and Chuck review how each interface will actually work with Richard. Richard can then either determine that a particular methodology will not work or adjust requirements to support the design. Richard can now flesh out the business and functional details of the rest of the application knowing how each piece will interact with others and how they will function as a unit.

Step 4 is the "final" requirements review with Chuck, Richard and Myles. At this time, all portions of the application will have been completely discovered and the functionality expected will be recorded. Chuck and Myles will go back into the cave to finalize the details of all interfaces. Everything must be covered.

Step 5 is assignment. Chuck divides sections of the application by pattern between Myles, Sharon, Mike and Noah. Patterns are assigned based on their inherent complexity compared to the experience of the developer. When a developer has not encountered a pattern they are responsible for, they will review the design documentation and templates for that pattern based on the platform (win/web) and proceed.

Step 6 is initial development. This stage of the application is more about data gathering than getting work done. Each team member is tasked with estimating how long an item of a specific pattern will take. He then actually develops and internally tests it until it is complete. When complete, he reports back to Chuck with the time it took. That developer should also understand the +/- factor of how long a more or less difficult implementation should change that base time estimate. It should also be expected that as the developer works more items, he will become more proficient at it. Sort of a software factory (heh).

Step 7 is real development. Chuck has baselines for all patterns for all developers. He can now plan how long the development phase will take with reasonable accuracy and granularity. Each developer knows how to do their job and only the skyline issues are raised to Chuck or Myles for decision making. When complete, each task is reported back (with development times) to Chuck who adjusts the schedule as appropriate. When a task is reported complete, Chuck will also review the average time that developer is taking to do that job and determine if the base estimates are still reasonable.

Step 8 is QA. Kelly understands the requirements and the patterns. She knows that there are very specific rules as to how the parts of the system can interact. Therefore, her first task is to smoke test the entire application looking for things that do not conform to known patterns. When she finds something, she tests this as early as possible in her cycle. This allows the highest risk portions of the application to get back to the developers for bug fixes as early as possible.

Kelly then continues to test the application in logical fashion and, when defects are found, they are directed to the developer who worked each one. The time to bug fix an item is added to the time estimate to perform the development of that pattern. Then, in future versions of that architecture, Chuck can raise a flag when a developer finished a given interface of a specific pattern much too early and QA it in development himself. The result may be that a developer is just that good or it may be that he is lazy.

Simple, no? Looks good on paper. The real trick here is getting everyone singing the same music at the same time. Well, that and figuring out what to do when you start with no development patterns at all...

On Architecture

When does the definition of a thing become more than the thing itself? Twenty years ago no one cared about software architecture. Now, when you review how the community defines architecture, its sometimes insightful and sometimes laughable. It is seldom consistent and often vague. It's a great title though.

The problem is that architecture is similar to art. When there is talent and thought involved, people notice. When you throw it against a wall and it sticks, it is not necessarily art (or architecture). Who defines it is as important as the definition.

So here goes...

When defined by its results, application architecture is successful when it:

1. Defines the patterns of all entities within a 'system'. This accomplishes improved project estimation and allows for specialization of development staff.
2. Defines specifically HOW each entity can interact with other entities and which entities cannot interact. This accomplishes loose coupling.
3. Defines WHEN each entity can communicate and what information is transferred during a transactional communication. This provides state management and requirements coverage in all application scenarios.

An entity is the realization of an atomic unit of the business problem that can be logically and clearly separated by its composition, function or derivation.

When all three of these objectives are met, the application’s architecture can be completely documented and translated into tangible components and base objects that enforce requirements within context.

... chew on that and let me know what you think. JJ