Category: Agile

Categories
Agile Business Intelligence

Business Intelligence – 3 Years of Agile

Last year at the PASS summit, I ran into someone and was discussing project management and Business Intelligence. They were so adamant that agile couldn’t work. And I had to correct them, as I have been doing agile now for three years at Trek in our BI group.

Yes, some things don’t exactly parallel to software development, but many things do. Sprints, Standups, estimations, stories, points, scrum master, releasing/delivering value on your iterations. And now even more with process like unit testing of SQL code and more, things are getting closer to software development in that regard.

I have an entire blog category dedicated to agile – more concepts but also talking about Business Intelligence teams in some of them.

Just remember, you can do BI and Agile, it works, and you can deliver a ton of value to the organization. Someone who might argue with you, well, they don’t know what they are doing in BI or in Agile, or the organization isn’t willing to change, which of course then it wouldn’t work.

Advice. Make sure you hold retrospectives, and make sure you make adjustments from whatever comes out of them.

Categories
Agile

Agile: Dealing with Your Bug Backlog

Previously I wrote about the types of bugs that might come up during a sprint and dealing with them. This post is more on the backlog of bugs your project may “acquire”.

Most software, if not all, has some kind of bugs. They crop up over time, causing users pain, maybe throwing errors, logic errors, but just overall just not letting your software do what it is trying to do.

After a while, you end up with a “bug backlog”. That is, of course, you aren’t fixing your bugs at the same or higher rate than they are coming in. If that is the case, then kudos!, but many applications end up having little nuances/bugs everywhere. Your backlog can grow quickly if you don’t do something about it. 100, 500, 1000+ bugs. What do you do?

There are two main things to do with your bugs. First off, prioritize them. 1, 2, 3, 4, 5, whatever scale you want. You need to separate the wheat from the chaff so to speak. You want to focus on the highest priority bugs first. Second, you want to make sure you have some kind of category for your bugs. Screen X, Screen Y, Workflow A, Workflow B, etc. That way, you can hone in on different areas of your application and “attack” those bugs. As an addition, you can maybe set the criticality of your bugs as well, but that isn’t completely necessary to start.

To Repeat: Prioritize and Categorize (and optionally Set Severity).

You should probably review new bugs on a weekly basis or some time interval to keep up with the bugs coming into your system. Some bugs are going to come in that are just end users using the system wrong (I always have Steve Jobs in my head saying “You are holding it wrong” here..) Then you have things you just might not care about fixing. You should close both of these bugs. Keep them in your bug system, but close them, and let the end users know what is up.

Other bugs that come through that are valid, should get the priority, category and severity ratings. You should also look at the existing backlog to see if there are existing bugs that are related or possibly duplicate. As an aside, I wish there were some better tools for doing this automatically, I have done some research and there are actually some scholarly papers on the subject. Maybe some systems have this built in, but the ones I have used havent.

You should try to keep your bug backlog down. Keep it groomed. The only way you are going to do that is to close bugs you aren’t going to fix, fix bugs that come in, and make your software more robust to avoid bugs in the first place. The latter is the hardest, but eventually you are going to have to fix some bugs.

There could be different approaches to fixing bugs. You could take one bug and make it one story, this is perfectly fine. You may take many bugs that are related, group them up and make one story, that works too. You may have a “hardening” sprint and just do bugs that sprint, maybe go the gamification route and add bug fixing bounties, there are endless options, but you are going to have to fix them, for a few reasons: to keep your software in tip top shape, and to keep your users happy.

If you end up ignoring your bug backlog for too long, you may just declare bug backlog bankruptcy. But once you start getting bugs in again, it will grow, you can’t keep ignoring it. It is like never taking the garbage out if you ignore it, it will keep piling up.

In the end, the best thing to do is come up with a systematic way to deal with your backlog, and stick with it. Make it a priority for everyone on the team to be conscious of the bugs and backlog and assist when needed in verifying bugs, finding dupes, entering bugs, fixing bugs, etc. Nobody likes buggy software, and software doesn’t like bugs either. It is a win-win.

Categories
Agile

Agile: Projects vs Support

One big questions that comes up ALWAYS when doing Agile is: “How do we deal with support requests (or issues, or helpdesk or operations)?”

Well, the answer, as you figured, is … It Depends.

First, it depends on what kind of project or team you have. Are you Development? Or Business Intelligence? or Creative/Design? or Infrastructure? Or XYZ?

One thing to think of as well is how is your organization structured? I went into a dev group and the developers couldn’t get anything done because customer service/support was constantly hitting them up with issues. What do you expect to happen? Magic? You need a buffer. 2nd Level support. Filter the issues so that there is just a trickle into “3rd Level” or Development.

You could even assign one dev to “3rd Level” for a sprint and reduce their velocity. But it always depends on how much is coming though. You want to reduce, reduce, reduce the noise coming into developers. Buffer.

What happens when there are issues that NEED to be looked into (server down, etc, etc). Well. Your “do’ers” need to look into it. You may have to pull out stories if it takes too long. Reduce the velocity. That is just the way it goes. If support issues continually cause you to pull out stories and reduce velocity, you should assess your organization structure, and get a support structure in place.

Big thing with support is this: YOU NEED TO TRACK STUFF. I can’t stress this enough. Bottom line is I have rarely or if ever seen stuff tracked well. This is a killer for your process. Issues need to be tracked for multiple reasons. Why?

Well, let’s look at a scenario or two.

1. Customer Calls Support
2. They work on issue for hours but never track anything
3. They go to 2nd level with the issue but since they didn’t track anything, who knows what was done already and what was changed, etc
4. 2nd level fumbles around for 3-4 more hours doing the same thing, but again, not tracking anything.
5. The issue gets handed off to 3rd level and it is a complete mess since nothing was tracked.
6. In the end, the issue might route back to 2nd level or whatever
7. In the very end, nothing was tracked at all. Even a Category or Sub Category, who the issue was for, what system, how much time, who worked on it, etc.

Now look at that scenario and think if everything was tracked. What if you could pull in all your support issues and Pivot them, slice and dice, see trends. Well, then you can figure out what you need for resources. Pretty simple actually. But harder in reality. People start working on something and run around like crazy and not tracking anything. It is a big problem.

Back to our main problem. Projects vs Support. Another thing everything depends on is this: Who is prioritizing your work? It is a main driver on what you work on. Someone needs to make a decision and say “This is a support issue, do it NOW, or.. This is not an issue, do it LATER, or.. this is part of our project, do a STORY” or something like that.

If the person who should be your main prioritizer doesn’t buffer or learn how to say NO, then everything because #1 top priority and everything you try to accomplish from a project perspective is worthless. Your prioritizer (if it is the Product Owner, your Manager, the Project Manager, whatever) can learn to buffer, and only let through the extra critical support issues to work on now, then you can dedicate 90-95% of your teams time to project (agile) work.

I think the main question of Projets vs Support really throws people off when it comes to Agile. Some groups are doing nothing but support so they are in a death spiral. They need a 2nd level support structure, or hell, even a 1st level support structure.

Teams that are trying to do Agile but feeling the pain of too many support issues, well they need a 2nd level support buffer.

Teams that are doing Agile but have 1st/2nd level support that isn’t tracking a ton are in need of some process control. (more so for resource management than anything).

As you can see, there are always ways to do things no matter what situation your team might be in, but definitely something you need to asess and figure out before you jump into a process.

Categories
Agile Geeky/Programming

Agile: Tech Debt Sprint

In software development, the biggest thing you can do for the users of your product is deliver value. Adding business value is critical to maintaining a good product and keeping users happy. But there is another group you need to keep happy as well, and that is your development team.

Technical Debt (think of it like real money debt) is the extra WTF’s that add up in your codebase over time. You add a new feature, and take some shortcuts, you say “we will fix it later” but later never comes. You end up with untested (Unit Tests/Developer Tests) code and with “legacy code” that is hard to maintain. How do you get rid of all that? You have to pay down your debt, that is where Tech Debt Sprints come in.

Just like paying down regular debt or even how people keep regular debt, you usually have some. People have mortgages, or car loans, that they pay down. So in any app you end up having tech debt, but you need to keep paying it down instead of just adding more and more debt, because sooner later you’d have to declare technical bankruptcy, which is no good. (Close Shop? Rewrite? etc).

What I like to do is balance tech debt in two ways.

1. First, in every sprint, there should be a percentage of points dedicated to technical debt. 15-25% is a good number.

2. Second, every X sprints you should flip that percentage. So if you usually do 70% enhancements, 10% bugs, and 20% tech debt, every 4-6 sprints, you should flip it around: 70% tech debt, 10% bugs, and 20% enhancements. I might most later on the concept of another type of sprint, a “Hardening Sprint” where you dedicate 70% to just bugs to “harden” your app.

Now, you may have to negotiate with your Product Owner, which after a while they might get high on the drug that is enhancements and business value. Most reasonably Product Owners though, will realize that the code and developers need to pay down tech debt to make the overall application better and also make it easier to get future enhancements to market faster.

You may even get some stories that I like to call “Trifecta” stories. What are they? Well it is that “magical” story that satisfies three causes, Enhancement, Bugs, and Tech Debt. An example might be a crappy custom control that you’ve had in your app that you can replace with built in functionality or a better architected 3rd party control. Users will get the added functionality (enhancement), crazy bugs from your custom code are gone (bug fixes) and tons of crappy code is just dropped from the codebase (tech debt). Everybody is a winner!

I blogged previously on using User Voice to track your tech debt items at a higher level and have devs vote on them, it works somewhat, better than nothing. Usually stories come out of each “idea”. The bottom line though is to make sure you dedicate time and effort to paying down your technical debt. Far too often dev shops ignore this or always put it off till some mythical “later” that never comes. You need to just do it. It will make your code and app better, your devs happy, and keep things moving forward.

Categories
Agile

Agile: Group Technical Review Worth It?

Every since I started practicing agile development and processes, I have lobbied for some type of group tech review before release. At the end of the sprint, the tech team (devs) get together and everyone can go over what they did, give some insight to what they did and why and everyone else can get a glimpse into the other work that was done. Tech reviews aren’t meant to beat anyone down, but they shouldn’t waste everyones time either. I think developers need to get that interaction as well as learning how to show what they did to their peers. Thoughts?

For the most part this idea of a sprintly tech review has worked well, but in others it breaks down.

Why? For one it takes time. If you are so crunched at the end of a sprint, then who has time to do a tech review? In some teams, this hasn’t been an issue, maybe it was better planning or estimating, or maybe just dumb luck, but doing the review made it into the sprint.

Maybe on some teams doing mini reviews with the group throughout the sprint instead of all at the end is the way to go. Maybe not doing any review at all.

My head is still with doing a group review at the end. I think the estimation and wrapping up at the end of a sprint probably would need to get looked at from a process perspective. What can be automated? I’m sure more than maybe you are doing now, etc.

I for one welcome new ways of doing things, but more efficient and still get the points across that need to be. Bring on the ideas – let me know your thoughts in the comments.

Categories
Agile

Agile: Day of Autonomy, 20% Time, etc

A about a year ago or so, I watched the RSA Animate short on YouTube,

httpv://www.youtube.com/watch?v=u6XAPnuFjJc

RSA Animate – Drive: The surprising truth about what motivates us

Go ahead, watch it, pretty good and just a little over 10 minutes.

One takeaway I got from the short was this: The “Day of Autonomy”. If I remember, they talk about a place where once a quarter or something they let everyone have a day of autonomy, to do whatever they want, but the goal is, at the end of that day, they need to show the group what they did. They saw people doing things that were probably never on any agenda or plan, but some things could help the company and spurred other ideas.

Google is infamous for their “20% time”, which has created things like Gmail, and more.

Just recently, I decided to give this a go in the Business Intelligence group, a day of autonomy. Our velocity for that sprint would be slightly lower, but I was intrigued to see what would come out of it.

The results are still being analyzed, but yeah, seeing some cool things. I think for the next one (if we do it), we will need to plan it a little better to get the results shown to everyone. We are a semi-distributed team, so there are some challenges there.

Overall though, I think the Day of Autonomy is probably a good thing, as long as it isn’t taken advantage of. The players on the team should just be bursting with ideas that they should have no problem coming up with something to do. If you don’t have ideas always swirling on something you could crank out or improve, then you probably need to start there and think a bit differently on how you approach your work.

To me, it almost adds on to the Agile way, being able to do things like this yet still adjust and be flexible, and get some great output on regular stories as well. Always keep improving.


Categories
Agile Business Intelligence Geeky/Programming SQLServerPedia Syndication

Agile: Creating an SSRS Burndown Chart Part 3

In the previous 2 parts (see Part 1 and Part 2) of this series I showed you how to get your data ready, and how to get your report started and your Datasets and parameters where you need them. In this part, we will get the graph functional, and in the next part, we will make it pretty.

Start by adding title to your report “Agile Burndown”, then add a Line Chart to your report. Make it somewhat big, delete the Chart Title and Axis Titles,  and remove the “Details” from the Category Groups. You should have something that looks like this:

 

image_thumb15

Now to get the data on and finish it off!

Drag your values over to your Chart Data Values area like this:

image_thumb[17]

One thing we need to tweak, and this is on the PointsLeft Value. Right click on the PointsLeft series and go to “Series Properties”. To the right of the Value field, click the Fx button (for Expression Functions).

We need to change this series to not write out anything to the graph if there are no points greater than today. Why? If you don’t do this, your graph line for PointsLeft will drop off to zero for dates in your sprint after the current day, and we don’t want this. This is what the expression should be:

 

=IIF(Sum(Fields!PointsLeft.Value)=0 And Fields!Date.Value > DateTime.Now,Nothing,Sum(Fields!PointsLeft.Value))

 

Pretty cool, your graph should actually work now and function as a working burndown chart. But of course we need to pretty it up! Look for the next and final post soon.

Categories
Agile Geeky/Programming SQLServerPedia Syndication

Agile: Creating an SSRS Burndown Chart Part 2

In the previous post in this series, Agile: Creating an SSRS Burndown Chart Part 1, I explained what data you would need to prepare to create an SSRS Burndown Chart (Sprint_Dates, Stories, Story_History). In this part of the series I will explain how to get a basic burndown report in SSRS.

First, fire up Report Builder 3.0 and create a new report (if the wizard pops up, just pick “Blank Report”). You need to add a Data Source to your report. In my example, I am just using a database on my localhost called Agile, so I connect to that and create a report Data Source.

image

 

We then need to add 3 Datasets to the report. (Burndown, Sprints, and CurrentSprint), and one parameter (Sprint) and we can then format our report.

 

Sprints (this will be a dropdown of Sprints for a user to choose from)

image

CurrentSprint (this will get the current sprint based on what day we view the report, default param for the Sprint parameter we will create)

image

 

For the Burndown, do the same thing, but since the query is so large, no screenshot, just the query:

;WITH DayHistory AS
(
SELECT
	 bd.[Date]
	,bd.PointsScheduled
	,bd.PointsLeft
	,bd.PointsScheduled - ((ROW_NUMBER() OVER (ORDER BY bd.[Date]) - 1) * (CAST(bd.PointsScheduled AS DECIMAL(15,6))/10.0)) AS 'Goal'
	,ROW_NUMBER() OVER (ORDER BY bd.[Date]) AS [DayNumber]
FROM (
	SELECT tot.Sprint,tot.LogDate AS [Date],
		CASE WHEN SUM(tot.PointsScheduled) = 0 THEN (SELECT SUM(Points)
		FROM dbo.Stories st
		WHERE Sprint = 'Sprint01') ELSE SUM(tot.PointsScheduled) END AS 'PointsScheduled',
		SUM(tot.PointsLeft) AS 'PointsLeft'
	FROM (
			-- Get History for the Current Sprint
			SELECT Sprint,LogDate,SUM(Points) AS 'PointsScheduled', SUM(PointsLeft) AS 'PointsLeft'
			 FROM
				 dbo.Story_History st
				WHERE Sprint = @Sprint
			GROUP BY Sprint,LogDate
			UNION
			-- Get the Current Day
			SELECT	Sprint AS 'Sprint',CAST(GETDATE() AS DATE) AS 'LogDate',
				SUM(Points) AS 'PointsScheduled',
				SUM(PointsLeft) AS 'PointsLeft'
				FROM dbo.Stories
				WHERE Sprint = @Sprint
			GROUP BY Sprint
			UNION
			-- Get zero's for all days in sprint to round out our dataset
			SELECT 'Sprint01' AS 'Sprint',WorkDate,0,0
			FROM dbo.Sprint_Dates
			WHERE Sprint = @Sprint
		) tot
	GROUP BY tot.Sprint,tot.LogDate
) bd
)
SELECT
	 a.[Date]
	,ISNULL(b.PointsScheduled, a.PointsScheduled) AS [PointsScheduled]
	,ISNULL(b.PointsScheduled, a.[PointsLeft]) AS [PointsLeft]
	,ISNULL(b.PointsScheduled, a.[Goal]) AS [Goal]
FROM DayHistory a
	LEFT OUTER JOIN DayHistory b
		ON a.DayNumber = b.DayNumber - 1
			AND b.DayNumber = 2
ORDER BY Date

 

This query is where all the magic happens. First, you need to get your story point values for the days, from your history, and also from the current day, you also need to get all days for that sprint with zero’s so that your graph will have all days and not just days with burndown. The CTE around the main query calculates the burndown by day so you end up with 4 columns, Date, PointsScheduled, PointsLeft, Goal

Now that you have your Datasets, we need to create a parameter, and then the graph!

Create a new parameter called “Sprint”, and set up the available values. Remember the Dataset we created to get all the sprints? Here is where you use it, like this:

image

Next, we want to setup the default values. Remember the query to get the “Current Sprint” – that is used to set our default.

image

Once you have that all setup, it is time to build the graph!

We are really close to having a working report here, and check back for part 3 of the series to get the graph working correctly, and part 4 for beautification!

Categories
Agile Geeky/Programming

Agile: Creating an SSRS Burndown Chart Part 1

The burndown chart. A must have for any ScrumMaster and Agile team. What it should show you is the rate at which you are “burning” down story points.

image

As you can see from the chart above, 3 lines. Red is your “points scheduled”, Green is the “goal” and blue is “points left”. While it is easy enough to create this chart and track the burndown manually in Excel, many teams after using Excel turn towards other systems to track their points and sprints. Right now I have one team using Unfuddle, one team using TFS, there are others that use this chart that use Footprints and really you can use whatever, and this chart can be built off of any database as long as it has the right data.

First, you need a table with your stories in it. You need to have some key columns – Sprint, Points and PointsLeft.

CREATE TABLE [dbo].[Stories](
	[Sprint] [varchar](50) NULL,
	[Points] [int] NULL,
	[PointsLeft] [int] NULL,
	[StoryId] [int] NOT NULL,
	[StoryText] [varchar](max) NULL
) ON [PRIMARY]

Now you may have others, like StoryId, StoryText, Assignee, etc but we aren’t concerned about those for this chart.

You then need at least 2 or tables, and a SQL job. 1 table to hold your Sprint and Dates and one to hold your “Story History”

 

CREATE TABLE [dbo].[Sprint_Dates](
	[Sprint] [varchar](50) NOT NULL,
	[WorkDate] [date] NOT NULL
) ON [PRIMARY]

CREATE TABLE [dbo].[Story_History](
	[LogDate] [date] NOT NULL,
	[Sprint] [varchar](50) NOT NULL,
	[Points] [int] NULL,
	[PointsLeft] [int] NULL
) ON [PRIMARY]

 

You will need a SQL Agent Job to run at 11:55 PM to capture the history, which should run this query:

 

INSERT INTO dbo.Story_History (LogDate,Sprint,Points,PointsLeft)
SELECT CAST(GETDATE() AS DATE),Sprint,SUM(Points),SUM(PointsLeft)
FROM dbo.Stories
GROUP BY Sprint

 

Remember you might not need all 3 tables, just the history and dates. You can get your actual stories off of wherever your stories are stored in the database. Now that you have your data in place, you can get ready to write the actual report! Look for the next part in this series.

Categories
Agile

Agile: Epics

So most people are familiar with “projects”. Something you do, have an end goal, usually a budget, resources, plan, etc. Sometimes you have projects within projects. Like a good example may be thinking of the next version of Microsoft Windows as the main project and revamping solitaire for the next version as the “epic”. Your end goal is to keep your main project moving forward, and do stories that apply to that project, but you also might run into mini projects along the way that could cross over numerous sprints, and you want to “group” like/similar stories together to form an “epic”.

What is nice about epics is they still let you do your sprints, and stories as normal,but you have a more higher reaching goal to get a set of stories complete to complete your epic. It also lets you look back and then see how many stories and points you did toward that epic and understand the effort,etc that went to completing it.

Use epics to group stories together to form a common goal for a similar “thing”. It will let you say “let’s focus on the blah epic” and you can make sure you get that piece of functionality done and track it and see all those stories together in the end.

P.S. Blogged from my iPad