Monday (7 miles easy)

7 miles? What? Oh yeah, mileage increases. My usual downtown route really maxes out at about 6 miles - anything more and you’re kind of just running in circles, which feels weird. So I went up through Maple Hill Cemetary, ran around Blossomwood Elementary, then dumped out in between the medical district and downtown. A couple of extra zigs and zags around Big Spring Park, and back to Five Points. 7 miles in the bag.

Unfortunately, my watch started to die on the last three miles, and I got a bit panicked. I ended up booking it the last few miles to try and finish before the battery died. Stupid. My knees or my metrics? Finished up at 9:36 min/mile, which is a bit too fast. Oh well.

Tuesday (4x1200 meters)

Last day of speed work. YAY!

It was hot and humid. Gross, really. 73 degrees when I started, but it felt like a wet sponge. I didn’t need this. On lap two, I started to feel like :poop:. I’ve had heat exhaustion a couple of times, and I was getting killed out there. No water either, which was probably a little stupid. I realized that if I wanted to finish this thing, I wasn’t going to do it by simply gutting it through.

I took my shirt off, sat down, and rested for a few minutes. Just getting my shirt off made a huge difference, and I cooled off relatively quickly. After a minute or two I felt like I could finish the workout, albeit rocking out my awesome programmer-bod.

Ran the next two iterations without a shirt on, and felt 1000% better. Ended up walking the recovery laps instead of running them at 11:00 min/mile, but being able to finish the last two speed intervals at full blast made me feel good. Or at least, better, since I didn’t want to puke any more.

Shirt back on, run back to house, finish up.

Wednesday

Alas, due to life creeping in (dog had to go to the vet), I didn’t manage to get any crosstraining in.

It’s very hard to get all of the workouts in, 7 days a week. 6 is actually doable - I can usually juggle morning committments such that I can push things to Wednesday or reschedule my runs to move my day off around. But it does mean crosstraining gets sacrified first, which is probably a bad idea.

Thursday (8 mile tempo)

The moment of truth. 1.5 mile warm-up/cool down. Did old marathon route, rocked it out. Even ran further since I hit the stop button on my Garmin at the turn around. What is up with me and my watch? Saw a coworker, that was cool. Lucky to live near routes, since he would have to had to drive to get there. Felt strong the whole way through, which was a confidence boost after the heat knock on Tuesday. Still a bit warm/humid today, but nothing quite look Tuesday.

When does fall arrive?

Friday (5 mile easy)

Downtown. Done.

Saturday (8 mile easy)

Looking at the schedule, I can see that I’m now up to 20 miles of easy running. This feels like an awful lot, but it is apparently working, as I’ve managed to keep up with the substance runs. I did find it hard to keep engaged during that entire 8 mile run at a 9:40 pace, but managed to complete it.

Sunday (10 mile long)

Given the previous week’s 15 miler, I found this run to be pretty easy, and thankfully so. I re-visited the old marathon course again, which is one of my defacto standard out and backs. At the end of the run, I caught sight of an albino squirrel, so that was pretty cool.

Totals

Distance: 49.05 miles

Time: 7:38:44

Calories: 6924

Albino Squirrels: 1

How

If you Google “engineering”, you’ll probably get the following definition:

The branch of science and technology concerned with the design, building, and use of engines, machines, and structures.

• The work done by, or the occupation of, an engineer.
• The action of working artfully to bring something about.

I particularly like the second part: the action of working artfully to bring something about.

As an engineer, we start with the How. In fact, we’re kind of obsessed with it:

• How do you perform your testing?

• How do you structure your software?

• How do you design your system?

• How do you express your requirements?

Which, of course, makes sense. Being an engineer is all about building something; if you do a crappy job of that, you’re not going to be a good engineer. And there’s a lot of good in honing your craft, becoming really, really good at software implementation. My favorite programming book is still Code Complete, and for good reason.

At some point, you’re very likely to be working on a software project when suddenly the project manager starts ratcheting up the pressure. Maybe a deadline slipped. Maybe there was no deadline but stakeholders are starting to get frustrated with the lack of a delivery. Maybe the software that was delivered didn’t contain nearly everything they thought they would get.

And that leads us to the next question.

What

As in, what the heck are we building anyway?

Time is the only finite resource. Every second of every day is a second you’re never going to get back. As I’m writing this, I’m choosing to write this instead of doing something else. In my case, it’s watching TV, so this is time well spent. But I could be working. I could be starting a side project. I could be reading something that furthers my skills, or learning a new language.

But I prioritized writing this (for some reason or another. Humans are strange). And that prioritization is immensely important in software.

A software project never really ends until the business producing it decides to stop selling it; even then, it can live on in some form. There will always be new bugs, features, improvements, refactorings, and more. The work to be done will always exceed the resources available. The order in which you do things thus greatly determines the likelihood of project success. Prioritization, more so than time estimates, requirements analysis, or anything else in the project management sphere of control determines the life or death of a software project.

As an engineer, you should care about this. No one cares how well you crafted a new feature if it wasn’t that important. And having your work be devalued sucks.

So let’s say we’ve got prioritization taken care of. We have a well groomed backlog, so we’re presumably building the right things in the right order. Great.

And then it gets time to launch, and you’re told that no one wants your software. The business doesn’t know how to sell it, or the business folks have realized that we can’t make sufficient money given the cost of customer acquisition. Perhaps the business doesn’t understand the sales model. Maybe the software doesn’t really solve a problem a customer is willing to pay for. Maybe the business thought they knew what the customer wanted, and no one actually confirmed it.

Or maybe there never was a customer.

When this happens, you’ll feel burned. And that leads us to the next question.

Why

Why are we building this thing at all?

That is:

• What problem are we solving?
• Is this a problem that we can charge money for?
• Does the customer who has this problem want to pay for a solution?
• How do we find these customers?
• How do we differentiate in the market?
• Can we survive as a business selling this?

Again: time is the most valuable resource. It’s one thing to save time by prioritizing your backlog, it’s another to make sure that you have the right backlog in the first place.

Some may argue that others get to make these decisions, and so an engineer doesn’t really need to know the Why or the What. “Focus on the How and stay in your lane.” And if you’re content with having your life wasted, you can do that. But great engineers don’t want to waste their lives building things that don’t matter. A great engineer will help the business course correct throughout the life of a project and make sure that the business’s resources are being allocated and spent in the best manner possible. A great engineer will have a valid opinion and a pertinent voice in the decision making process of why are we building this thing.

A novice engineer cares about the How.

A good engineer focuses on the What.

A great engineer understands the Why.

This week was going to be a challenge. AstriCon.

I don’t usually get a lot of sleep during AstriCon, and I typically work my liver out more than any other part of my body. Not exactly the best set up for marathon training, particularly when one’s training plan is based on cumulative fatigue. Going in, I came up with the following plan:

1. Get to bed early Monday night. I wouldn’t get in until nearly midnight, which actually worked a little bit in my favor: less time to get pulled into whatever conference vortex was forming. I planned to move quickly through the hotel lobby and get to bed immediately.

2. Get my speedwork day in on Tuesday, as per plan. This was the first of the hard runs.

3. Move my Friday easy-ish day to Wednesday, as opposed to taking it off. I had an early flight out on Friday, and this way I didn’t have to worry about running before the flight.

4. Suck it up enough to get up on Thursday. Wednesday night is usually the most active night, so this would be a challenge.

Monday (5 miles easy)

I did my five miles easy downtown. A few hours at work, and it was off to the airport for an evening flight to Orlando. I had to wait for a few other folks once I landed. Combined with the taxi ride over, I didn’t get to hotel until midnight, and fell asleep about 12:30.

No worries. Six hours of sleep is enough, right?

Here we go.

Tuesday (5x1000m speed work)

The alarm went off at 6-ish. Groggy, I got myself up, out, and going.

As I got going, I noticed that it smelled a bit. Kind of sulfurish. I probably shouldn’t have been surprised; Florida is kind of one big swamp. It was also hot and humid. Knew that the route was mostly a big square, figured I’d run to one of the far ‘corners’ and turn around. Out and back. Went fine until I saw the hill. WTF?

That’s a 100 ft. elevation change, and it goes up a lot faster than the chart shows. By the time I hit the last interval I was sucking wind, and sucking wind hard, but I did it.

FU Floridia. You stink, literally.

Wednesday (6 miles easy)

On Wednesday I woke up and ran with a group of fellow AstriCon attendees - one a coworker who was also training for the marathon, and another who is a longtime Asterisk community member and all around great guy. He was training for his first marathon, so the two of us who had done the insanity before had a nice chat with him as we ran the same route I had the day before.

I don’t usually run with groups, but when you’re running with good people, it’s quite enjoyable. Good conversation when needed, quiet when we’re all heading up that silly hill. I can see this would be a bit weird on a tempo day or other substance day, but easy days are great for group running. I did have to them to slow down a bit, no faster than 9:40-ish for me please. Luckily, both of them were will willing to humor me.

All together, we finished up at 9:50 min/mile. Nice.

Thursday (8 mile tempo Sleep)

Guess what happened the night before?

Conference Matt. Conference Matt happened.

Really, it wasn’t a lot of boozing, but way too much talking. I didn’t get back up to the room until 12:30 or so, and wasn’t really asleep until 1. When I woke, there was just no way I was getting out there. Could I have done it? Maybe, I wasn’t hung over. But running after drinking with no sleep didn’t sound very smart. Rationalizing it to myself, I thought, “I know, I can juggle it again - we’ll run Friday evening when I get home. That still gives me Saturday for an easy run, with my long on Sunday. YAY.”

As it turns out, this was not the best of plans.

Friday (8 mile tempo 6 mile tempo)

My flight was early in the morning, so there was no chance of getting in the run early. The plane trip back was uneventful, getting me back to good old HSV early in the afternoon. I don’t like running in the evenings, but damnit, I was going to get all my runs in this week.

Got home; talked to Katie.

“You know it’s going to rain all weekend, right? I don’t see you doing 15 miles on a treadmill on Sunday.”

“Shit.”

Stupid hurricane Nate.

If I moved the 15 mile run to Saturday morning, I should’t do my tempo run; no back-to-back substance runs. Or else I could cancel my long run. I didn’t like either of those options, so I went with a compromise:

1. Drop the 8 mile tempo to a 6 mile tempo. Still a substance run, but slightly easier.

2. Do the 15 miles tomorrow, but don’t set a 9:20 min/mile pace. Instead, do 5 at slow pace, 5 at long pace, and finish it up with 5 at slow pace.

And so I went and did the tempo run and did not enjoy it. How do people run in the evening? All during the run my lunch wanted to come out and regale me with tales of my intestinal tract. Maybe those burger sliders with chili weren’t a great idea. Somehow, I got it done with a solid ~8:32 min/mile pace. No pain in the knees at least.

Saturday was going to be fun.

Saturday (15 mile long)

WHoooooooooooooooooooo boy.

Getting up before the crack of dawn, I drove down to Whitesburg Elementary to start the loop of transcendence. Bailey Cove. 5 miles or so of flat, straight-to-only-slightly-curved road, before you loop and head up the interminable Chaney Thompson for more of the same.

Y.A.Y.

Even without the Garmin warning me, I quickly got up to long distance pace. “What the hell,” I thought, “it isn’t hurting.” I kept it at that pace for the whole 15. Unlike previous long runs, I never had to stretch out the legs to keep the knee from hurting.

Training seems to be working. I’m pretty sure my knee pain was from weak quads, and the Hanson plan appears to be fixing that.

Finished well on it, 9:25 min/mile.

Sunday (6 mile slow)

Hurricane Nate was a bit of a soggy mess. Lots of rain that seemed to alternate between sprinkling and spattering. The real stuff, with the wind, was still several hours away, and I hate treadmills. Running in the rain it is.

No big deal, finished at 9:44 min/mile.

Totals

Distance: 47.95 miles
Time: 7:28:43
Calories: 6794
Rules broken: 1

Monday (Off…?)

Ever since I moved to Alabama, I’ve wanted to hike the Walls of Jericho. Somewhere near Scottsboro there’s a sign for it (only 20 miles away!), and every time I’ve driven by it I’ve thought to myself, “I should do that.” A few friends warned me that the hike back up is a bit “strenuous”, but that if you were in okay shape, it was doable.

And hey, I’m training for a marathon. I’m in okay shape, right?

Katie let me know that there was a guided hike on Monday, which provided the perfect excuse to finally go and experience the Walls. Yes, it meant taking a day off of work, and I’d need to juggle my runs accordingly. Looking at my schedule, I figured I’d skip my usual crosstraining (hiking is crosstraining, right?) and move that from Wednesday to Monday. That would mean at least 8 straight days of running. Yikes.

Whatever, you’re only 36 once.

The hike was totally worth it. The pictures don’t really do it justice.

While it would have been great to swim in the pool at the bottom of the walls, being on a guided hike with a large group made me feel a bit awkward. Plus, nobody else was doing it, and I didn’t really pack anything for that. Eating lunch on the cliff overlooking the pool was good enough.

Once I’d had lunch, it was back up. And up. And up. I left the guided group in the dust and power hiked it out, and it was still two hours of constant climbing. My iPhone says I did 7.7-ish miles of hiking, and climbed 109 flights of stairs. It certainly felt like it. My right knee was a bit sore afterwards, in that oh so vague “complaining” sense. This wouldn’t bother me, but this was supposed to be my “off” day. Hopefully everything holds together. We’ll see how the substance runs go.

Tuesday (6x800m speed work)

While I was a bit sore from the hike, this ended up being no big deal muscle/joint wise. Frustratingly, my warm up lap was screwed up. I have my workout set up to let me do a 1.5 mile warm up/cool down, triggered by lap button presses. Somehow, I had set this one up wrong, and it immediately clicked over into the speed interval at mile 1, causing me to have to do the first 800m/400R on a hill and not on a track. Luckily, I think as the distance has gone up I’ve actually found it a little bit easier to do the speed intervals. I think it comes from not having to ratchet the effort up and down quite so often. Even so, the laps were still pretty brutal, the last two in particular. I’m not sure if I’ll feel the same way on the 5x1000m runs in Orlando next week. By the end of the substance run, my knee was a bit sore, but not twinging.

I’ll call that a victory.

Wednesday (6 mile slow)

Took this one extra slow, given that my knee was still a bit sore. Had no problems, and it was a nice, slow, pleasent run. Later in the afternoon, I still had no knee soreness, so things seemed to be healing up just fine.

Thursday (6 mile tempo)

The run I was fearing, as intensity + longer distance is usually my recipe for disaster. Knee was fine, although it felt “present”. I’m not sure how else to describe it - it isn’t sore, it isn’t painful, I can just feel its existance more than usual. As normal, tempo runs feel like the hardest run in the set, given how tired my legs are. Usually, I have my crosstraining day off in between to “freshen up” for this run; even missing it, I finished just fine. As I was starting my last tempo mile, I wondered if right now, I could run another three miles. I get to find out next week when I bump up to an 8 mile tempo run.

Friday (5 mile slow)

A nice easy loop around downtown, run at roughly a 9:41 min/mile pace. Easy peasy.

Saturday (6 mile slow)

Another nice easy loop around downtown. I was particularly proud of this because I ran it slow, giving my legs extra rest. Finished it up at 10:03 min/mile. I never thought I’d be proud at holding myself at roughly a 10:00 min/mile pace, but sometimes controlling yourself is harder than just simply letting loose.

Sometimes, it’s a lot harder.

Sunday (12 mile long)

As this would be my last 12 mile long before AstriCon - which has traditionally been the death of training (more on that in the next post) - I was anxious to get in a solid long run. Plus, my mileage was going to go up next week with an 8 mile tempo run and 15 mile long, and having a strong long run to finish off the week would give me a nice mental boost.

Unfortunately, I hadn’t gone more than maybe two or three minutes before I encountered a house on fire.

Seriously.

Okay, so not a big fire, but a fire nonetheless. I mean, it’s 6:15 AM in the morning, there’s smoke coming from the porch, and actual flames. (Ironically, as I approached the house, I smelled the classic smell of burning wood and thought to myself, “wow, that’s nostalgic”, only to realize that the wood was their house.)

Holy flirking shnit.

I knocked on their door; no answer. Pounded, shouted, nothing. Not surprising, really. Who wants to answer their door at 6:15 to a guy wearing bright clothes and super-short shorts?

Ran around the house, looking for a hose. None found.

Here then, we have a moral dilemma. I have water on me, about 18 ounces. I need that water. I’ve got 12 miles to go, it’s the South, and it’s unseasonably warm. There’s no water fountains along my route, no public places to refill.

I hesitated.

With an internal sigh, I dumped half of each bottle onto the fire, hoping it would put it out. I got maybe a quarter of the fire to go out, but then the wind picked up and it flared to life again. Feeling the remainder of the water in my bottles, I decided that:

• Dumping the rest of the water on the fire wouldn’t solve the problem.
• I could theoretically do 12 miles on half the water I usually bring.

Time to call in the professionals. It felt weird to try and call the fire department for a fire that was technically less than 1 ft x 1 ft, but I was running out of options. Alas, there wasn’t a non-emergency number for the fire department (and, really, is any structure fire an emergency? Presumably yes), so 911 it was.

It was a very awkward 911 call.

“What is your emergency?”

“Well, it’s not really an emergency, but there is a house on fire.”

“Sir, are you in the house?”

“It’s not my house, I was running by and saw it.”

“Are there any occupants trapped?”

“Well, I can walk around the fire and pound on the door, so I don’t think so. But they aren’t answering their door either.”

“…”

Anywho, the fire department showed up, put the fire out in 10 seconds, and thanked me for my service. I waved, and took back off out of my neighboorhood on my course.

15 minutes behind.

This is a problem, as I tell Katie how long I think I’m going to be out and about when I’m doing my long runs so she doesn’t worry. I always give myself a good 30 minutes of padding in case I have to walk some, and I’d just eaten 15 minutes into that. If for whatever reason I had to slow down considerably, I’d be bumping pretty close into my time limit.

Don’t. Panic. The. Wife.

Even so, I was good so long as I could keep myself at my 9:20 min/mile pace. As I ran, things felt… good. I had run 6 days in a row, and while my legs were sore, after the first mile things just clicked into place and I bumped along. After the first three or four miles it was hard to keep myself from running too fast. No knee soreness, other than the usual “you’re running on me for 12 miles” kinds of soreness that one can expect.

I finished up with a solid 9:22 min/mile and got home in plenty of time. Yay.

Totals

Distance: 44.53 miles
Time: 7:01:06
Calories: 6305
Fires: 1

I wrote a rather lengthy blog post for the Asterisk Blog about the recent RTP related security vulnerabilities in Asterisk. I won’t go into more detail here than I did there, but given that there was a public disclosure of flaws in the first security release that occurred without warning to the project team, things were not as smooth as we would have liked.

To find vulnerabilities, you have to build tools and perform actions that in a non-controlled environment would clearly be malicious. That’s not bad in and of itself; what you do with those tools and the information you gain from them determines your ethical bent.

If you look at some of the major vulnerabilities, you can see that things have gone a bit “commercial”. Vulnerabilities now get names, marketing sites, hell - even stickers. The researchers who found the RTP vulnerability in Asterisk quickly piggy-backed on this, dubbing the vulnerability “RTPBleed”.

I get that researchers need to get paid, and marketing certainly is important for those who are trying to run a business. But as a business, do you owe the projects and products that you find vulnerabilities in anything? Or do you only owe your customers who paid you to find the vulnerabilities? How do those conflicts get resolved?

I sure don’t know, and I doubt I have a good, well formed opinion on it. But there’s something slightly sour when websites and Github projects go up quickly, and the developers responsible for delivering the working software are unaware of the full impact of the vulnerability they supposedly fixed.

To keep things from getting long and repetitive - and because I don’t want to make that many tables - I’m going to stop putting all the metrics into these updates. I’ll wait for something impressive before throwing in all that detail…

Monday - Easy (5 miles)

Monday was a nice easy 5 miler, which I ended up completing at roughly a 9:51 pace. I’m finding that even though I’m a little fatigued from the substance runs, I quickly end up bumping against the maximum easy pace. I’ll probably need to be more conscious of my pacing for some easy runs, as I really should run some of them closer to a 10:00 min/mile to 10:20 min/mile pace. Anyway. I did notice that my hips were a bit sore afterwards, but I chalk that up to all the walking at the Monte Sano Art Show.

As an aside, I’ve always found it odd that if I walk 6 miles, my hips will want to straight up murder me, but if I run 12 miles, they don’t even make a squeak. Body mechanics be weird, yo.

Tuesday - Speed Work (8x600 meters)

Ah, speed work. I suspect that by the time I’m done with the speed work section of Hanson’s, this will be my least favorite day of training. Or tempo work. It remains to be seen.

The 1.5 mile warm up went just fine, and I made my way over to the local high school track. After two of the 600 meter intervals, I didn’t feel super great. When I ran the speed work last week, Hurricane Irma had just passed by and the day was cooler, even if it was humid. Not so this Tuesday; it was warm and humid, and I was feeling it. I kept glancing nervously at the ridgeline/treeline, waiting for the sun to peek out and start baking me. As I ran those endless loops, I lost count on which iteration was on. At one point I thought I had done six intervals; looking at my watch, I saw that I was actually only halfway. How the hell I lost count between four and six I have no idea, but F-bombs began to rain down onto the track. By the time I legitimately had hit six laps, nausea was pretty constant. At the same time, the sun peeked out from the treeline, starting my inevitable steaming/baking. Slightly panicked, the last two iterations became a blur. I was so damn happy when the Garmin beeped on the last interval and I could jog a mile and a half home. By the time I reached the house, I felt okay-ish, but I’m worried about next week. Even more so for the 5x1000m, which is going to be in Orlando.

Humidity sucks.

Thursday - Tempo Run (6 miles)

So this is really an 8 mile run, with 6 miles run at tempo and 2 miles run at easy pace. I was supposed to do five miles, but in week 9 this jumps to 8 miles at tempo pace, and I didn’t want a 3 mile jump. Knowing myself, that may do me in; if not physically, then psychologically. On top of that, I had a massage the night before. While I love them, and view them as being incredibly beneficial in the long term, they do usually make me more sore for a few days afterwards. Starting out I was a bit stiff, and the first tempo mile didn’t feel very good. It doesn’t help that there are several rolling hills on the route I chose. The first being near the end of mile 2, another half way through mile 3, and then a nasty one at about 2/3 of the way through mile 7.

After running the first three tempo miles (half-way, yeah!), I was getting pretty sluggish and tired. My refueling plan usually calls for water every two miles and something with energy every four miles, so I popped four Gu gel blocks. I’m still not sure if I like the blocks or the gels better. To their benefit, the packages for the blocks can be opened before a run, which means I’m not desperately trying to tear open a plastic package with my teeth while running. As those went down, I had a nice straight-ish section, which helped me to get some energy back. By the time I hit the mile 7 hill, I didn’t feel all that terrible. Not great, but not terrible. If I had to run two more tempo miles, I probably could have done it. The last easy mile was easy (as it should have been), although things felt like they were cramping up a bit as I finished. Overall pace was 8:50 min/mile, with my tempo pacing hold between 8:30 and 8:36.

Friday - Easy Run (5 miles) / Saturday - Easy Run (6 miles)

Both the easy runs were easy, which is what they should be. The first was done at a 9:43 min/mile pace, the second at a 9:41 min/mile pace. At some point I’m going to need to slow down even further some of these easy runs. For me, the book recommends some easy runs to be run at ~9:40 min/mile and some at ~10:20 min/mile, and I’m definitely erring on the side of faster right now. As the substance runs gets harder, running some of these firmly in the “recovery” category is going to be important. And a challenge: it’s getting hard to run slower than 10:00 min/mile.

Sunday - Long Run (12 miles)

This was supposed to be a 10 mile long run, but I bumped this up to 12 miles based on my previous long runs. Things held together shockingly well until about the last three miles, at which point I started to feel my knees. Concerned, I tried to compensate by engaging my quads more consciously, which seemed to work. I never had a knee twinge during the long run, but things were a bit more sore in the knees than I’d like afterward. Maybe bumping the mileage on the long run wasn’t a good idea? I’m not sure. On the one hand, increasing mileage too much and running too fast/hard is how you injure yourself. On the other hand, that giant bump in two weeks scares the willies out of me, and this really was just a 3 mile bump over the previous week.

A larger concern for me after this long run is a hike I had planned at the Walls of Jericho tomorrow. I’ve never been there before, and I’ve heard rumors that the hike out is long and uphill. I’m going to push my Monday easy run to Wednesday, moving my “off day” to Monday and count the hike as “cross-training”. This is probably a bad idea, but I’ve been wanting to do this hike for the better part of a decade, and taking advantage of a free group hike is probably the only time I’d get to do it. So… here’s hoping my knees withstand a couple hours of downhill/uphill hiking.

Totals

Distance: 45.29 miles
Time: 8:07:24
Calories: 6782
F-Bombs: > 10

This week was the first substantive week of marathon training. Prior to this week, I had built up to running five days a week:

• A ‘fast-ish’ day of running, where I would attempt to run most of the run at approximate marathon pace.
• 4 to 5 miles of hills, generally running 2 to 2.5 up Monte Sano and back down
• (Rest)
• Medium run day, generally between 8 and 10 miles run at a slow pace (but probably not slow enough)
• 5 mile easy run day
• Long run day, generally between 10 and 15 miles

There’s nothing necessarily wrong with this, as it incorporates some tempo like runs, hill work (strength training), and two long-ish running days. The downsides are:

• Something of a lack of science to my method. I came up with the plan by incorporating elements of my previous marathon attempts (which haven’t gone well) and a lot of reading about various methods, including Hanson and Pfitzinger. That’s probably not a smart idea.
• Pacing. Looking at my run results, I’ve been running a bit too fast in most places.
• For the longest distance I’m currently doing, my volume still probably isn’t enough.

Enter the Hanson Marathon Method.

I explained in my previous post why I chose it. So how did it go for me?

Monday - Speed Work (12x400m repeats)

I moved the speed work up from Tuesday, it’s regularly scheduled day, to Monday as I had an early morning meeting Tuesday and hadn’t run on Sunday, thereby avoiding the “no back-to-back SOS workouts” rule. That weekend I purchased a pair of Nike Zoom Fly shoes from our local Fleet Feet store, and I was pretty excited to try them out. Due to being a Linux user, I’ve had some trouble with the Garmin Connect app pushing workouts down to my watch, so I decided to do my run at a local high school track, which conveniently was within a 1.5 mile-ish warm up run from my house.

The plan was:

• 9:40 min/mile warm up/cool down
• 7:50 min/mile 400m sprints with 11:00 min/mile 400m recover laps

This being my first set of speed work, I think I did relatively well. The splits aren’t exactly the best, as I found I’d usually rocket off to a too fast start on each 400m interval and have to throttle it back to get to the right pace. This led to most of my 400m repeats being run too fast. My recovery intervals were worse, and I eventually just decided to try and go no faster than a 10:20 min/mile. Next week I should have the workout pushed to the watch, and the beeps/buzzing of the Garmin should keep me in line better.

After writing all of this down, I realize I am still running way too fast. Drat.

Tuesday - Easy Run (5 miles)

At this point, I had managed to get my workouts downloaded to my watch, which made pacing a lot easier. I did my favorite route through downtown Huntsville, which includes a nice jaunt down from the square through Big Spring park. I also swapped shoes for my Brooks Glycerin 15’s, my old stand-by in running shoes.

While I run this route a lot, the fact that it goes through some nice neighboorhoods and downtown makes it a favorite. It certainly beats running alongside interstate access roads or down a four line thoroughfare, both of which are some of my other options.

Wednesday - Crosstrain

I’ve chosen to do some strength training at a local gym for my crosstraining. I find that keeping my upper body in “okay-ish” shape helps with long runs. Without it, after about 10 miles I usually end up with some ice-pick like pain jabbing between my shoulder blades. I may write someday what this routine looks like; suffice to say it takes about 60 minutes and isn’t liable to turn me into He-Man any time soon.

Thursday - Tempo (5 miles)

I’ve read a lot about the dreaded tempo runs. Up until now, other than a few ill advised “push it” moments, I’ve usually stayed above the 9:00 min/mile mark. If I’m going to get a 3:45 marathon time, however, I have to be able to hit and hold a 8:30 min/mile pace. That feels ambitious, but not out of reach. Since my runs are now starting to include some warm up/cool down times, I decided to add those to all of my workouts and push that to my watch as well.

When Thursday dawned, my legs felt tired. I suspect this was the after effects of having run my speed work a bit faster than I should have, combined with the start of my “cumulative fatigue”. Or it’s too early and I’m just imagining how I’m going to be feeling. Anyway.

Two things I learned from this run:

1. I kind of messed up my tempo run slightly, as I had set my goal tempo pace to 8:35 as opposed to 8:30. Rather than suffer five miles of beeping and buzzing, I just went with it. I’ll fix it up for the tempo run next week.
2. That last mile at tempo pace was no bueno. I had to keep on mentally telling myself “one more mile, you’re good; one more mile, you’re good”. And by mentally, I mean I was muttering that out loud through some slightly random neighboorhood. I’m not worried that I won’t be able to do another 5 mile tempo run next week; I am worried about week 9 when I go from a 5 mile tempo run to an 8 mile tempo run. That’s a huge leap. I may try to gradually increase this over the next two weeks just to even out the pain a bit.

Friday - Easy (5 miles)

After my tempo run, I woke up on Friday feeling pretty good. I again decided to go for about a 9:40 min/mile pace, and did the good old downtown course again.

Note that I’m including my small-ish warm up/cool down routines for easy runs now too, because why not. I’m moving damnit, and those calories count.

Saturday - Easy (6 miles)

Saturday morning I felt a bit more tired than on Friday morning. That didn’t surprise me; I usually feel the worst after effects of a strenuous exercise two days later. I was a bit concerned, as I had volunteered to do parking at the Monte Sano Art Show all day that day, and I knew I was going to be on my feet for around 8 hours at least. I decided to shoot for anything over a 10:00 min/mile pace. I still wanted to finish up in about an hour, as I had to be up in the state park by 8:00 AM or so to start my volunteer duties.

Sunday - Long (10 miles)

Sunday morning dawned and I knew I was in a bit of trouble. All of the walking yesterday had definitely made me pretty sore, despite my best efforts the night before on loosening things up on a foam roller. My largest concern was not that I wouldn’t be able to run 10 miles, but that my soreness might cause me to whack out my knee and cause larger damage. I decided that at the first sign of trouble I’d stop and stretch and/or walk, as I’d rather take a DNF on my first long run than mess up the whole shebang.

Although, I will admit in a moment of honesty, that if I mess up my knee on week 6 of a program that is supposed to help prevent injury then I’m probably not ready to do this thing in the first place. Anyway. Let’s keep that little realistic thought out of the way for now.

Despite my misgivings, it went just fine. I hit up the old Rocket City marathon course. Starting in and around downtown, it forms a nice out and back through South/Central Huntsville, keeping mostly to small quiet neighboorhoods. Given the course’s popularity, you’re always bound to run into a couple dozen other runners, which I like on long runs (safety first). I kept up my long run pace of ~9:20 min/mile relatively well. I also practiced taking a Gu gel while running; something I haven’t done in the past. I’ve always walked my water/Gu breaks, but if I’m going to keep up a pace throughout an entire marathon, there’s no time like the present to train for it.

Total Impressions

• Total Distance: 42.78 miles
• Total Time: 409.38 minutes
• Total Calories: 6014

Well, it is just week one, so I probably should tamper my enthusiasm.

1. My legs feel pretty good today (which was an Easy run day, but that’s technically on week 2). I definitely have a perma “ache” or soreness in them, but it’s that good level of soreness/achiness that says that you’ve done something, not any kind of acute pain. I can imagine that as the weeks go on and the mileage goes up, this is only going to become “more of a thing”. I’m okay with that.
2. I hate getting up in the morning and running, but once I get out the door I’m fine. It’s worse when it’s dark out; I’m finding myself hitting the snooze once or twice just so I can leave just after 6 when there’s a hint of light as opposed to 5:45-ish when there isn’t. It’s a weird mental thing. I’m not looking forward to having the cold pile up on top of that.

I ran a marathon once.

I was 29, and the notion of kicking off my third decade with a marathon appealed to me. Up until that point in time, I think I had run two half marathons and was in reasonable-ish shape; reasonable-ish because I have an aversion to running in the cold (even Alabama cold) and generally slack off during the winter months. The Rocket City Marathon, held in December, seemed ideal. I’d train during the summer/fall, and would be able to run the race when it is good and cold in North Alabama, but not have to train too long in below 40 temperatures. I gave myself what I thought was plenty of time, and like so many marathon newbies, I picked a Hal Higdon Beginner marathon plan and started training. I’m pretty sure I used the Beginner Two plan, as I recall having two 20 mile runs. On that fateful second 20 mile run, my knee twinged pretty badly and I ended up having a lot of pain after I completed the run. Concerned, I backed off on the training some, thinking that if I gave my knee some time, I’d be okay by race day. To test it, I went ahead and ran the Huntsville Half Marathon, doing fairly well with a 1:43:20. This was stupid on two fronts: first, running a race on a slightly bum knee is never a good plan. Second, I naively assumed that I could just sort of “double” that half marathon time and tack on another 10 minutes or so for my goal marathon time.

Not surprisingly, while I managed to finish, the marathon did not go super great.

This was on the old course of the Rocket City Marathon. At the time, the race had an infamous stretch along a straight, boring, four lane road (Bailey Cove Road) that went on for five or six miles. Once you were done with that transcendent experience, you had to come up another slightly inclined straightaway that is Chaney Thompson. I remember feeling pretty good when I started that stretch; by the time I was halfway up it, I remember not feeling very good. My knee twinge flared. That caused me to slow down. Then it flared - and stayed flared. Sharp, stabbing pain. By mile 18 I was hobbling. Hobbling very, very slowly. The notion of shuffle walking another 8 miles did not appeal to me, but because it was my first marathon and I did not want a DNF to grace my entrance into my 30s, I gutted it out.

Finishing was still an emotional experience, even if I crossed the finish line in 4:09:27 - a far worse result than my unrealistic 3:40. Worse than my miserable finish was what that marathon did to my running. I fell off the running bandwagon hard, and have never really managed to get back to that same level of effort.

I have tried a few times over the past six years. At least twice, I’ve started ramping up my mileage, with the thought in the back of my head that I would avenge my Rocket City Marathon debut. Both times, I’ve ended up hurting myself; usually, again, stabby knee pain that won’t go away. The last attempt at this was roughly three years ago. Since then, I have managed to finish a trail 25k (not without a hydration/electrolyte related mishap, but that’s another problem) as well as another decent half marathon finish. Even so, the allure of finishing a marathon strong has appealed to me. I don’t really care that much how long it takes, I just want to finish it strong.

(That being said, finishing under 4 hours would still be nice, if for no other reason than I don’t want to be out there pounding pavement that long.)

Earlier this year, I began to consider bookending the first half of my thirties with a marathon. I started to run again in April, and have since slowly built up my endurance running long, slow mileage. Longest run so far has been 15 miles, but I’ve been hitting 10 - 14 mile long runs with a weekly run total of over 30 miles since July. So: can I run a marathon before November 25th?

Nope.

In reality, I’m still not quite there. There’s only so many marathons in the Southeast that I’d want to run before November. For me to be ready for a marathon by early November, I’d have to be further into a real training program than I am right now, and “pushing it” for the sake of an arbitrary date is clearly not a good idea.

How about the Rocket City Marathon?

Tracing back from where I am right now, that’s 13 weeks out. Most beginner marathon training programs are 16 to 18 weeks, and with my currently weekly mileage being where it is, this feels more doable.

So how to train?

After a lot of reading on reddit, I settled on the Hanson Beginner Marathon Plan. I even bought the book on Amazon - a first for me. I settled on this approach for a few reasons:

1. The Hanson Method doesn’t sugar coat it: running a marathon is hard. In fact, if anything, the author almost seems to try and discourage you slightly: training for a marathon is a commitment, and it takes a lot of time. Given my last few attempts, I’d rather go with a realistic - but successful - approach.

2. The very long runs have been my Achilles Heel. Two out of the past three attempts, I’ve hurt myself on a 20 mile run. I’m fairly sure my other injury also occurred not long after an 18 or 20 mile attempt. Running longer than 3 hours in training has gone poorly for me, and the Hanson Method addresses this head on. A lot of people focus on the promise of only running 16 miles; however, they miss the author’s reason for limiting the long run. Generally, for most people, running longer than 3 hours is damaging on the body. Doing so in training is likely to lead to injury. For those of us in the 9 - 10 min/mile camp, 16 miles is roughly a 3 hour training run; hence that limit. This is a plan that addresses issues in my previous training attempts.

3. The plan helps with pacing. Like most running neophytes, I tend to run too fast and end up hurting myself as a result. The plan gives you pacing guidelines, which for the slow runs is far slower than I was running.

4. The focus on speed work. I’ve never really known how to do speed work properly. I had attempted to do fartleks in the past, but it didn’t feel “right”. I’d run fast either too little or too much, and burn out on what felt like a very inconsistent run. Eventually, I just stopped doing speed work. The Hanson Method has you doing speed work for several weeks, followed by what it calls “strength” runs, consisting of longer iterations run at slightly faster than marathon pace. Again, the author explains very well why these substance runs (Something Of Substance, or SOS) matter, and how to view them in relation to your easy/recovery runs and long runs.

Since my mileage was already high, and the first five weeks of the Hanson method are really building up to the first set of SOS runs, I started out on week six of the plan this week.

We’ll see how it goes…

This is the second - or maybe third? - time I’ve rebuilt this blog. Over the next few days I’m going to pull in some of my posts from my old WordPress site.

Rather than using a heavy weight blogging platform again, I’ve instead decided to go with Hexo, a static site generator. Why?

1. Posts can be written in text in markdown. I vastly prefer this over yet another WYSIWYG HTML/RichText editor.

2. All of the site and the content can be managed using Git.

3. Connecting and deploying the site is trivial using Netlify.

Posts will be sporadic, as per usual.

Houston, We Have a Problem

“21055537: the number of failed authentication attempts against a public #Asterisk 12 server since January 11th” - @joshnet

— Matt Jordan (@mattcjordan) July 16, 2014

Yeah… that’s not cool.

When Josh told me how many failed authentication attempts his public Asterisk 12 server was getting, I wouldn’t say I was surprised: it is, after all, something of the wild west still on ye olde internet. I enjoy the hilarity of having fraud-bots break themselves on a tide of 401s. Seeing the total number of perturbed electrons is a laugh. But you know what’s more fun? Real-time stats.

I wondered: would it be possible to get information about the number of failed authentication attempts in real-time? If we included the source IP address, would that yield any interesting information?

Statistics are fun, folks. And, plus, it is for security. I can justify spending time on that. These people are clearly bad. Our statistics will serve a higher purpose. This is for a good cause.

For great justice!

Stasis: Enter the message bus

We’re going to start off using our sample module. In Asterisk 12 and later versions, the Stasis message bus will publish notifications about security events to subscribed listeners. We can use the message bus to get notified any time an authentication fails.

Getting Stasis moving

I’m making the assumption that we’ve copied our res_sample_module.c file and named it res_auth_stats.c. Again, feel free to name it whatever you like.

1. First, let’s get the right headers pulled into our module. We’ll probably want the main header for Stasis, asterisk/stasis.h. Stasis has a concept called a message router, which simplifies a lot of boiler plate code that can grow when you have to handle multiple message types that are published on a single topic. However, in our case, we only have a single message type, ast_security_event_type, that will be published to the topic we’ll subscribe to. As such, it really is overkill for this application, so we’ll just deal with managing the subscription ourselves. Said Stasis topic for security events is defined in asterisk/security_events.h, so let’s add that as well. Finally, the payload in the message that will be delivered to us is encoded in JSON, so we’ll need to add asterisk/json.h too.

   1 2 3 4

   #include "asterisk/module.h" #include "asterisk/stasis.h" #include "asterisk/security_events.h" #include "asterisk/json.h"

2. In load_module, we’ll subscribe to the ast_security_topic and tell it to call handle_security_event when the topic receives a message. The third parameter to the subscription function let’s us pass an object to the callback function whenever it is called; we don’t need it, so we’ll just pass it NULL. We’ll want to keep that subscription, so at the top of our module, we’ll declare a static struct stasis_subscription *.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   /*! Our Stasis subscription to the security topic */ static struct stasis_subscription *sub; ... static int load_module(void) { sub = stasis_subscribe(ast_security_topic(), handle_security_event, NULL); if (!sub) { return AST_MODULE_LOAD_FAILURE; } return AST_MODULE_LOAD_SUCCESS; }

3. Since we’re subscribing to a Stasis topic when the module is loaded, we also need to unsubcribe when the module is unloaded. To do that, we can call stasis_unsubscribe_and_join - the join implying that the unsubscribe will block until all current messages being published to our subscription have been delivered. This is important, as unsubscribing does not prevent in-flight messages from being delievered; since our module is unloading, this is likely to have “unhappy” effects.

   1 2 3 4 5 6 7

   static int unload_module(void) { stasis_unsubscribe_and_join(sub); sub = NULL; return 0; }

4. Now, we’re ready to implement the handler. Let’s get the method defined:

   1 2 3 4 5

   static void handle_security_event(void *data, struct stasis_subscription *sub, struct stasis_message *message) { }

   A Stasis message handler takes in three parameters:

   • A void * to a piece of data. If we had passed an ao2 object when we called stasis_subscribe, this would be pointing to our object. Since we passed NULL, this will be NULL every time our function is invoked.

   • A pointer to the stasis_subscription that caused this function to be called. You can have a single handler handle multiple subscriptions, and you can also cancel your subscription in the callback. For our purposes, we’re (a) going to always be subscribed to the topic as long as the module is loaded, and (b) we are only subscribed to a single topic. So we won’t worry about this parameter.

   • A pointer to our message. All messages published over Stasis are an instance of struct stasis_message, which is an opaque object. It’s up to us to determine if we want to handle that message or not.

5. Let’s add some basic defensive checking in here. A topic can have many messages types published to it; of these, we know we only care about ast_security_event_type. Let’s ignore all the others:

   1 2 3 4 5 6 7 8

   static void handle_security_event(void *data, struct stasis_subscription *sub, struct stasis_message *message) { if (stasis_message_type(message) != ast_security_event_type()) { return; } }

6. Now that we know that our message type is ast_security_event_type, we can safely extract the message payload. The stasis_message_payload function extracts whatever payload was passed along with the struct stasis_message as a void *. It is incredibly important to note that by convention, message payloads passed with Stasis messages are immutable. You must not change them. Why is this the case?
   A Stasis message that is published to a topic is delivered to all of that topic’s subscribers. There could be many modules that are intrested in security information. When designing Stasis, we had two options:

   • (1) Do a deep copy of the message payload for each message that is delivered. This would incur a pretty steep penalty on all consumers of Stasis, even if they did not need to modify the message data. Publishers would also have to implement a copy callback for each message payload.

   • (2) Pretend that the message payload is immutable and can’t be modified (this is C after all, if you want to shoot yourself in the foot, you’re more than welcome to). If a subscriber needs to modify the message data, it has to copy the payload itself.

     For performance reasons, we chose option #2. In practice, this has worked out well: many subscribers don’t need to change the message payloads; they merely need to know that something occurred.

     Anyway, the code:

     1 2 3 4 5 6 7 8 9 10

     struct ast_json_payload *payload; if (stasis_message_type(message) != ast_security_event_type()) { return; } payload = stasis_message_data(message); if (!payload || !payload->json) { return; }

     Note that our payload is of type struct ast_json_payload, which is a thin ao2 wrapper around a struct ast_json object. Just for safety’s sake, we make sure that both the wrapper and the underlying JSON object aren’t NULL before manipulating them.

7. Now that we have our payload, let’s print it out. The JSON wrapper API provides a handy way of doing this via ast_json_dump_string_format. This will give us an idea of what exactly is in the payload of a security event:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

   static void handle_security_event(void *data, struct stasis_subscription *sub, struct stasis_message *message) { struct ast_json_payload *payload; char *str_json; if (stasis_message_type(message) != ast_security_event_type()) { return; } payload = stasis_message_data(message); if (!payload || !payload->json) { return; } str_json = ast_json_dump_string_format(payload->json, AST_JSON_PRETTY); if (str_json) { ast_log(LOG_NOTICE, "Security! %s\n", str_json); } ast_json_free(str_json); }

Stasis in action

Let’s make a security event. While there’s plenty of ways to generate a security event in Asterisk, one of the easiest is to just fail an AMI login. You’re more than welcome to do any failed (or even successful) login to test out the module, just make sure it is through a channel driver/module that actually emits security events!

AMI Demo

Build and install the module, then:

1
2
3
4
5
6
7
8
9
10
11
12
13

$ telnet 127.0.0.1 5038
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Asterisk Call Manager/2.4.0
Action: Login
Username: i_am_not_a_user
Secret: nope
    
Response: Error
Message: Authentication failed
     
Connection closed by foreign host.

In Asterisk, we should see something like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

*CLI> [Aug  3 16:00:51] NOTICE[12019]: manager.c:2959 authenticate: 127.0.0.1 tried to authenticate with nonexistent user 'i_am_not_a_user'
[Aug  3 16:00:51] NOTICE[12019]: manager.c:2996 authenticate: 127.0.0.1 failed to authenticate as 'i_am_not_a_user'
[Aug  3 16:00:51] NOTICE[12008]: res_auth_stats.c:60 handle_security_event: Security! {
    "SecurityEvent": 1,
    "EventVersion": "1",
    "EventTV": "2014-08-03T16:00:51.052-0500",
    "Service": "AMI",
    "Severity": "Error",
    "AccountID": "i_am_not_a_user",
    "SessionID": "0x7f50ae8aebd0",
    "LocalAddress": "IPV4/TCP/0.0.0.0/5038",
    "RemoteAddress": "IPV4/TCP/127.0.0.1/57546",
    "SessionTV": "1969-12-31T18:00:00.000-0600"
}
  == Connect attempt from '127.0.0.1' unable to authenticate

Great success!

Dissecting the event

There’s a few interesting fields in the security event that we could build statistics from, and a few that we should consider carefully when writing the next portion of our module. In no particular order, here are a few thoughts to guide the next part of the development:

• There are a number of different types of security events, which are conveyed by the SecurityEvent field. This integer value corresponds to the enum ast_security_event_type. There are a fair number of security events that we may not care about (at least not for this incarnation of the module). Since what we want to track are failed authentication attempts, we will need to filter out events based on this value.

• The Service field tells us who raised the security event. If we felt like it, we could use that to only look at SIP attacks, or failed AMI logins, or what not. For now, I’m going to opt to not care about where the security event was raised from: the fact that we get one is sufficient.

• The RemoteAddress is interesting: it tells us where the security issue came from. While we’re concerned with statistics - and I think keeping track of how many failed logins a particular source had is pretty interesting - for people using fail2ban, iptables, or other tools to restrict access, this is a pretty useful field. Consume, update; rinse, repeat.

STATS!

Let’s get rid of the log message and start doing something interesting. In Asterisk 12, we added a module, res_statsd, that does much what its namesake implies: it allows Asterisk to send stats to a StatsD server. StatsD is really cool: if you have a statistic you want to track, it has a way to consume it. With a number of pluggable backends, there’s also (usually) a way to display it. And it’s open source!

In the interest of full disclosure, installing statsd on my laptop hit a few … snags. Libraries and what-not. I’ll post again with how well this module works in practice. For now, let’s just hope the theory is sound.

To use this module, we’ll want to pull in Asterisk’s integration library with StatsD, statsd.h.

1
2
3
4

...
#include "asterisk/json.h"
#include "asterisk/statsd.h"
...

And we should go ahead and declare that our module depends on res_statsd:

1
2
3
4

/*** MODULEINFO
    <support_level>extended</support_level>
    <depend>res_statsd</depend>
***/

Since we’re not going to print out the Stasis message any more, go ahead and delete the char *str_json. Now that we know we’re getting messages, let’s filter out the ones we don’t care about:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

static void handle_security_event(void *data, struct stasis_subscription *sub,
                                  struct stasis_message *message)
{
    struct ast_json_payload *payload;
    int event_type;
    
    if (stasis_message_type(message) != ast_security_event_type()) {
        return;
    }
    
    payload = stasis_message_data(message);
    if (!payload || !payload->json) {
        return;
    }
    
    event_type = ast_json_integer_get(ast_json_object_get(payload->json, "SecurityEvent"));
    switch (event_type) {
    case AST_SECURITY_EVENT_INVAL_ACCT_ID:
    case AST_SECURITY_EVENT_INVAL_PASSWORD:
    case AST_SECURITY_EVENT_CHAL_RESP_FAILED:
        break;
    default:
        return;
    }
}

Here, after pulling out the payload from the Stasis message, we get the SecurityEvent field out and assign it to an integer, event_type. Note that we know that the value will be one of the AST_SECURITY_EVENT_* values. In my case, I only care when someone:

• Fails to provide a valid account

• Fails to provide a valid password

• Fails a challenge check (rather important for SIP)

So we bail on any of the event types that aren’t one of those.

The first stat I’ll send to StatsD are the number of times a particular address trips one of those three security events. StatsD uses a period delineated message format, where each period denotes a category of statistics. The API provided by Asterisk’s StatsD module lets us send any statistic using ast_statsd_log. In this case, we want to just simply bump a count every time we get a failed message, so we’ll use the statistic type of AST_STATSD_METER.

Using a struct ast_str to build up the message we sent to StatsD, we’d have something that looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

static void handle_security_event(void *data, struct stasis_subscription *sub,
                                  struct stasis_message *message)
{
    struct ast_str *remote_msg;
    struct ast_json_payload *payload;
    int event_type;
    
    if (stasis_message_type(message) != ast_security_event_type()) {
        return;
    }
    
    payload = stasis_message_data(message);
    if (!payload || !payload->json) {
        return;
    }
    
    event_type = ast_json_integer_get(ast_json_object_get(payload->json, "SecurityEvent"));
    switch (event_type) {
    case AST_SECURITY_EVENT_INVAL_ACCT_ID:
    case AST_SECURITY_EVENT_INVAL_PASSWORD:
    case AST_SECURITY_EVENT_CHAL_RESP_FAILED:
        break;
    default:
        return;
    }
    
    remote_msg = ast_str_create(64);
    if (!remote_msg) {
        return;
    }
    
    ast_str_set(&remote_msg, 0, "security.failed_auth.%s.%s",
    ast_json_string_get(ast_json_object_get(payload->json, "Service")),
    ast_json_string_get(ast_json_object_get(payload->json, "RemoteAddress")));
    ast_statsd_log(ast_str_buffer(remote_msg), AST_STATSD_METER, 1);
    
    ast_free(remote_msg);
}

Cool! If we get an attack from, say, 192.168.0.1 over UDP via SIP, we’d send the following message to StatsD:

1

security.failed_auth.SIP.UDP/192.168.0.1/5060

Except, we have one tiny problem… IP addresses are period delineated. Whoops.

(cleaner) STATS!

We really want the address we receive from the security event to be its own “ID”, identifying what initiated the security event. That means we really need to mark the octets in an IPv4 address with something other than a ‘.’. We also need to lose the port: if the connection is TCP based, that port is going to bounce all over the map (and we probably don’t care which port it originated from either). Since the address is delineated with a ‘/‘ character, we can just drop the last bit of information that’s returned to us in the RemoteAddress field. Let’s write a few helper functions to do that:

1
2
3
4
5
6
7
8
9
10
11
12
13

static char *sanitize_address(char *buffer)
{
    char *current = buffer;
    
    while ((current = strchr(current, '.'))) {
        *current = '_';
    }
    
    current = strrchr(buffer, '/');
    *current = '\0';
    
    return buffer;
}

Note that we don’t need to return anything here, as this modifies buffer in place, but I find those semantics to be nice. Using the return value makes the modification of the buffer parameter obvious.

That should turn this:

1

IPV4/TCP/127.0.0.1/57546

Into this:

1

IPV4/TCP/127_0_0_1

Nifty.

Since we used a struct ast_str * to build our message, we’ll need to pull out the RemoteAddress into a char * to manipulate it.

REMEMBER: STASIS MESSAGES ARE IMMUTABLE.

We’re about to mutate a field in it; you cannot just muck around with this value in the message. Let’s do it safely:

1
2
3
4
5
6

char *remote_address;
    
...
     
remote_address = ast_strdupa(ast_json_string_get(ast_json_object_get(payload->json, "RemoteAddress")));
remote_address = sanitize_address(remote_address);

Better. With the rest of the code, this now looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

static void handle_security_event(void *data, struct stasis_subscription *sub,
                                  struct stasis_message *message)
{
    struct ast_str *remote_msg;
    struct ast_json_payload *payload;
    char *remote_address;
    int event_type;
    
    if (stasis_message_type(message) != ast_security_event_type()) {
        return;
    }
    
    payload = stasis_message_data(message);
    if (!payload || !payload->json) {
        return;
    }
    
    event_type = ast_json_integer_get(ast_json_object_get(payload->json, "SecurityEvent"));
    switch (event_type) {
    case AST_SECURITY_EVENT_INVAL_ACCT_ID:
    case AST_SECURITY_EVENT_INVAL_PASSWORD:
    case AST_SECURITY_EVENT_CHAL_RESP_FAILED:
        break;
    default:
        return;
    }
    
    remote_msg = ast_str_create(64);
    if (!remote_msg) {
        return;
    }
    
    service = ast_json_string_get(ast_json_object_get(payload->json, "Service"));
    remote_address = ast_strdupa(ast_json_string_get(ast_json_object_get(payload->json, "RemoteAddress")));
    remote_address = sanitize_address(remote_address);
    
    ast_str_set(&remote_msg, 0, "security.failed_auth.%s.%s", service, remote_address);
    ast_statsd_log(ast_str_buffer(remote_msg), AST_STATSD_METER, 1);
    
    ast_free(remote_msg);
}

Yay! But what else can we do with this?

MOAR (cleaner) STATS!

So, right now, we’re keeping track of each individual remote address that fails authentication. That may be a bit aggressive for some scenarios - sometimes, we may just want to know how many SIP authentication requests have failed. So let’s track that. We’ll use a new string buffer (dual purposing buffers just feels me with ewww), and populate it with a new stat:

1
2
3
4

service = ast_json_string_get(ast_json_object_get(payload->json, "Service"));

ast_str_set(&count_msg, 0, "security.failed_auth.%s.count", service);
ast_statsd_log(ast_str_buffer(count_msg), AST_STATSD_METER, 1);

Since we’re unlikely to get a remote address of count, this should work out okay for us. With the rest of the code, this looks like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

static void handle_security_event(void *data, struct stasis_subscription *sub,
                                  struct stasis_message *message)
{
    struct ast_str *remote_msg;
    struct ast_str *count_msg;
    struct ast_json_payload *payload;
    const char *service;
    char *remote_address;
    int event_type;
    
    if (stasis_message_type(message) != ast_security_event_type()) {
        return;
    }
    
    payload = stasis_message_data(message);
    if (!payload || !payload->json) {
        return;
    }
    
    event_type = ast_json_integer_get(ast_json_object_get(payload->json, "SecurityEvent"));
    switch (event_type) {
    case AST_SECURITY_EVENT_INVAL_ACCT_ID:
    case AST_SECURITY_EVENT_INVAL_PASSWORD:
    case AST_SECURITY_EVENT_CHAL_RESP_FAILED:
        break;
    default:
        return;
    }
    
    remote_msg = ast_str_create(64);
    count_msg = ast_str_create(64);
    if (!remote_msg || !count_msg) {
        ast_free(remote_msg);
        ast_free(count_msg);
        return;
    }
    
    service = ast_json_string_get(ast_json_object_get(payload->json, "Service"));
    
    ast_str_set(&count_msg, 0, "security.failed_auth.%s.count", service);
    ast_statsd_log(ast_str_buffer(count_msg), AST_STATSD_METER, 1);
    
    remote_address = ast_strdupa(ast_json_string_get(ast_json_object_get(payload->json, "RemoteAddress")));
    remote_address = sanitize_address(remote_address);
    
    ast_str_set(&remote_msg, 0, "security.failed_auth.%s.%s", service, remote_address);
    ast_statsd_log(ast_str_buffer(remote_msg), AST_STATSD_METER, 1);
    
    ast_free(remote_msg);
    ast_free(count_msg);
}

In Conclusion

And there we go! Statistics of those trying to h4x0r your PBX, delivered to your StatsD server. In this particular case, getting this information off of Stasis was probably the most direct method, since we want to programmatically pass this information off to StatsD. On the other hand, since security events are now passed over AMI, we could do this in another language as well. If I wanted to update iptables or fail2ban, I’d probably use the AMI route - it’s generally easier to do things in Python or JavaScript than C (sorry C afficianados). On the other hand: this also makes for a much more interesting blog post and an Asterisk module!