By In Football

The Art of Officiating

Here’s something a bit odd to think about: You really can tell how important officiating is to a sport by how many officials the sport uses. For instance, in tennis there can be up to 11 officials in professional matches — I think the breakdown is one chair umpire, four sideline judges. four service line judges, and two baseline judges — PLUS a referee and a chief umpire, PLUS an advanced camera system called Hawk-eye to be the final word.

Golf, meanwhile, has plainclothes rules officials who never seem to be anywhere nearby, who ride around in golf carts and wear Secret Service ear-pieces connected to some 24-hour all-rules radio station or something. They never see anything, and they pop-up every now and again, mostly to tell Tiger Woods he basically can do whatever he wants.

This gets at the difference in the two sports. In golf, for the most part, the golfers are on the honor system. Yes, these days there are millions of officials watching their every move on television and they somehow know how to call in their complaints.* But generally speaking golfers are supposed to officiate themselves and their partners.

*This is actually pretty amazing to me. It usually takes me an hour to find a number for my cable company when the Internet goes out or to call Apple when something goes wrong, but these people know how to find the telephone number to the rules people at a professional golf tournament. Crazy.

Meanwhile, at the highest level of tennis, they don’t trust players to do ANYTHING themselves. They have people watching to see if the player’s foot happens to touch the line while serving. They have someone to MAKE the call, and they have someone to OVERRULE the call if necessary, and they have an appeal system that allows players to take their case to the highest court in the sport, a complex, insanely expensive computer system that blends the angles of six cameras and is displayed by a not especially realistic looking animation of a tennis ball and a line.

So you could say tennis considers officiating more important than golf does.

If anything, the contrast is even more stark in the two footballs. We talked a bit about this on the Poscast this week — yes, we’re back — but it came up again Monday night. In soccer, there are technically four officials. There is a referee, two linesmen and a fourth official who is essentially in charge of substitutions, keeping up with time stoppages and getting yelled at by managers because he happens to be standing close by. The fourth official is important, I suppose, in the same way that Commissioner Gordon is important in Batman. He is not really a major player.

So, basically. an entire soccer match is officiated by three people, and the referee — the one official actually on the pitch — runs up and down the pitch like a madman and makes the bulk of the important decisions, including fouls, penalties, who to book with a yellow or red card and so on and so on. It is a near impossible task, of course, which I think has an enormous impact on how soccer is played. For instance, players tend to dive quite often in soccer — more, probably, than any of the other major sports in America. I think this is because players know that it’s likely that the referee is not close enough to get a great angle of the play. The dives can look quite comical (or infuriating, depending on your point of view) on replay. But live, from 20 meters away, with a view partially obscured, it can look like a foul, and the players know this. So the diving continues.

I think soccer matches — especially in the Premier League — tend to be officiated in a more literary way, not unlike the way someone officiates a wedding. By that I mean that the rules of soccer seem to be viewed as interesting guidelines meant to make the game more dramatic and fun rather than strict laws that are to be studied and preserved and argued about by talmudic scholars.

You can tell this is true by the way soccer matches end. They almost never end when something interesting is happening. While most American sports are timed to end at the precise 100th of a second, soccer ends whenever the official decides it should end. And this is almost NEVER when a team is close to scoring a goal. When a team is attacking in an interesting way, the referee seems to be thinking, “Hey, sure, I know, the match should be over, but I want to see this.”

I find it hilarious at the end of regulation time when that fourth official holds up that sign that shows how much stoppage time is left in the match — it’s always a single number like “4.” Yep, four minutes left. Exactly four minutes? Well, uh, maybe not exactly four, you know, it’s kind of a rough estimate. So is it really 4:28? Is it 4:58? Is it 8:49? Nobody’s saying — (according to my soccer-expert editor Randy the rule is that AT LEAST four minutes must go by). Let’s just call it “4,” and think about the particulars later. Don’t worry, The referee will keep looking at his watch and tell you when it’s over.

Meanwhile, American football referees will stop the game for a half an hour waiting for the timekeeper to put 2:27 back on the clock when it’s showing 2:26.

So, that’s soccer — one referee, two lines judges and a sport that is officiated in a grand, over-arching way where no one seems to get too worked up about the minutia. There’s a great Latin expression about trivial points of law: de minimis non curat lex — “The law does not concern itself with trifling matters.” That’s how soccer is officiated.

And then — there’s football, where there is no such thing as a trifling matter. There are seven officials on the field in a pro football game — a referee, an umpire, a head linesman, a line judge, a back judge, a side judge and a field judge. There is also a replay official in the booth. There are also numerous former officials on retainer for the networks to ask questions.

And in pro football, there is nothing too small to consider. Nothing. The clock is supposed to stop and start precisely.* First downs are measured to the nanometer. Catches are reviewed from 10 different angles. Touchdowns are played back and forth like the Zapbruder film — did the tip of the ball cross the goal line before the knee touched a blade of grass?

*I don’t know if you noticed this — probably not — but during the Panthers-Patriots game, the clock did not seem to start on time on a play with 2:45 left. It was probably only stuck for one or two or three seconds — but that really might have made all the difference. Because it was stuck, the Panthers ran a play at 2:01 and the clock stopped for the two minute warning. If not for that delay, the Panthers would have run that play AFTER the two minute warning, which could have had pretty significant impact on the clock. The Patriots ran their final play with three seconds left — they might have had to run something completely different and earlier if the clock had been wound properly. This is what I mean by trifling matters.

We expect crazy precision in football officiating — all the more crazy because there is nothing precise about pro football officiating. The difference between holding and legal block is microscopic and in the eye of the beholder. The difference, between pass interference and defensive holding and illegal contact and a clean play is even smaller. Here’s something to think about: On punt and kickoff returns, you often will have some sort of penalty on the receiving team. A hold. A block in the back. A hands to the face. Whatever. There seems to be a penalty on maybe 30% of all returned kicks. You know this and have probably complained about it at some point.

But, think about it another way: You almost NEVER have a penalty on the kicking team. Yes, now and again you will have an offsides or someone illegally crunching the guy calling for the fair catch, but generally speaking you must be allowed to do ANYTHING YOU WANT if you are the kicking team. That’s football. What’s illegal for one guy is not illegal for another. What’s a late hit for one official is not late for another. A holding call in the first quarter isn’t necessarily a holding call in the second quarter. The game is a whirlwind of subjectivity. It is like trying to referee a earthquake. Still, we as fans have an almost unlimited capacity for parsing the rules.

And so we go to Monday Night, where Carolina led New England by four points, last play of the game. Tom Brady dropped back, then stepped up in the pocket, then threw the ball into the end zone where it was intercepted by Carolina’s Robert Lester. Behind the ball, New England’s tight end Rob Gronkowski was obviously and blatantly being kept from the football by Panthers’ linebacker Luke Kuechly.

Back judge Greg Meyer threw a flag that was obviously going to be pass interference. Then there was a discussion, and referee Clete Blakeman announced that there was no penalty on the play and the game was over.

What was most interesting was not the call/no call, nor was it the glorious Bill Belichick grumpathon press conference that, at some point, sounded to me like a Saturday Night Live skit.

Reporter 1: Bill, um, did the officials give you any explanation for that last play?
Belichick (looking like he wanted to strangle reporter): No.
Reporter 2: How about the official: Did he give you an explanation?
Belichick (looking like he wanted drop reporter from top of tall building): No.
Reporter 3: On that controversial last play, did you get some kind of clarification of what the officials saw?
Belichick (looking like he wanted to feed reporter to sharks): No.
Reporter 4. Bill, was there some kind of statement from the official expounding on the final play?
Belichick (looking like he wanted to run over reporter with car five times): No.
Reporter 5: Bill, what was the interpretation given to you on that last play by the official?
Belichick takes reporter into side room and drops him into dungeon where he is eaten by lions.

No, what was most interesting was that long after the play, heck even now, people STILL do not know if that was really pass interference. The rules are that opaque. The defining question seems to be whether or not the ball was catchable. ESPN’s referee on call Gerry Austin said the ball was definitely not catchable. Other people who seemed to have actually watched the video thought it was certainly catchable had a linebacker not been holding Gronkowski back. ESPN’s analysts ran a delightful simulation of the play with Steve Young as Tom Brady and Trent Dilfer as Rob Gronkowski. Twitter decided it was a mess. Referee Clete Blakeman released a statement saying they made the right call because, well, they just did. It was a fun free for all.

Personally, I think it was pretty clearly pass interference — and I say this as someone who was happy to see the Panthers win. I also think it wouldn’t have mattered at all if Greg Meyer doesn’t throw the flag. If he doesn’t throw the flag, the game ends, and only a few hardcore fans and GIF makers would have noticed Kuechly mauling Gronkowski. Anyway, a soccer referee would have dealt with it differently. He would have just let the Patriots run another play.

42 Responses to The Art of Officiating

  1. Michael Blouse says:

    Good stuff. Horse raving always baffles me when races are listed at ‘about 1 mile.’ Hmmm, might be helpful to know. Or not. If it’s just more than a mile, my horse fades. If it’s just less than a mile, my horse is coming at the wire.

  2. Michael Blouse says:

    Horse racing. I’m usually raving, tho.

  3. steve says:

    on the point about football (soccer) refs ending the game, the board at the end of the game indicates a minimum of 4 minutes added time so if the clock runs over 4 mins it usually means someone has wasted time during the added on time. Referees also dont blow for full time until the ball is in an unthreatening area because there was a goal in a world cup in 1974 (i think) which was disallowed because the ref blew the whistle whilst the ball was heading in, and there was uproar.

  4. Wilbur says:

    I’ve only seen replays of the play, but why wasn’t defensive holding called? Or considered?

    • Adam says:

      If it had been in the spirit of soccer, defensive holding would have absolutely been called. The 5 yard penalty combined with the untimed replay of the down due to the game not being allowed to end on a defensive penalty would have been the perfect do-over for the ambiguous situation.

      Unfortunately, the ball was clearly in the air when the contact started. Once the ball is in the air, pass interference is the only option aside from manufacturing some kind of hands to the face penalty.

  5. dglnj says:

    The soccer referee doesn’t make “the bulk of the important decisions”; the soccer referee makes ALL the decisions. The assistant referees do not make decisions; they make indications to the referee, to assist him in making decisions. Even the things that you usually see an assistant referee signal (which team gets a throw-in, offside, goal) are indications to the referee that he decides.

    In soccer, the referee controls the game based on a small set of general laws. In football, seven officials administer the game based on a large set of extremely specific rules.

    • Jake Bucsko says:

      Not only that, dglnj, but they don’t even have to announce what call they have made! In the most recent World Cup, Maurice Edu had a goal disallowed against Slovenia because…reasons. Imagine if in the playoffs, Peyton Manning threw a 40 yd TD pass into traffic, only the ref threw the flag and didn’t say why. Pass interference? Holding? Sorry, we don’t have to tell you. Ever.

      As for the game last night, I can see how it wouldn’t be pass interference, but it certainly should have been defensive holding. The Patriots still would have had one play from the 13 yd line, which likely would have failed, but the refs missed that one.

      P.S. How dare you suggest that Commissioner Jim Gordon is anything but essential to the Batman mythos! I demand you retract this statement.

      • Cuban X Senators says:

        I love this exoticization of soccer refereeing on the Edu goal too (see below on how the end of soccer is remarkably like basketball & American football).

        The whistle that killed the play on which Edu put the ball in the net was exactly like a call you see all the time in basketball when one referee make an awful call to give one team the ball & then immediately makes a phantom traveling call to reverse the bad call. This is precisely what happened v. Slovenia. The US was given a free kick on a spurious call, (the AR before whom the foul was called I’m sure communicated that) & as soon as the ball was kicked the whistle blew.

        Nearly as many basketball games have such events as don’t, but it happens in a soccer game Americans watch & they’ve never seen the like.

    • Chris Harman says:

      What about offside calls and fouls that take place right in front of the assistant referees? They are either swatting at flies when they raise their flags or we are nitpicking about semantics.

      • dglnj says:

        An AR’s signal is an indication to the referee. The referee may make a call based on the AR’s indication or he may not; it’s entirely up to the referee. You can call it “nitpicking about semantics”, but it’s essential to the way the game is officiated – only the referee makes decisions.

        So in football where the officials are all empowered to make calls, with the Referee being a sort of “first among equals”, you get the back judge throwing a flag because he sees a penalty and then three or four officials huddling up to discuss what they all saw and reach a decision as to what the correct call should be. In soccer, you get the AR indicating a foul – but the referee has sole discretion to decide whether or not to call a foul.

        This is Law Five (the best law there is): “The decisions of the referee regarding facts connected with play are final.”

  6. BigSteve says:

    Joe, your Latin is rusty. It’s ‘de minimis non curat lex.’

  7. Brent says:

    Gronk is a great player, but he cannot break the laws of physics, no way he could have been able to get to a ball that underthrown, his momentum was going away from the ball, even without the contact by the Panthers’ LB, so no pass interference. However, I am on board with a defensive holding call, which moves the ball to the 13 yard line and gives the Patriots an untimed play from there. (and yes holding can occur with the ball in the air, it is illegal contact that cannot occur after the ball is thrown)

    • Adam says:

      I didn’t got to the original rulebook, but here is a page that quotes the rules in breaking down that play discussing why holding can’t be called. While the website address looks biased, I didn’t find the analysis to be so.

    • While you’re right that the holding rule doesn’t explicitly say anything about the foul being committed while the ball is in the air, the language in the rule book clearly states that once a quarterback “demonstrates no further intention to pass the ball,” a parenthetical example of which is having already thrown a forward or backward pass, the restrictions on defensive holding will end. Brady had released the ball before Kuechly impeded Gronk’s route, so there was no further intention to pass the ball — thus, no defensive holding. Given the timing of the play’s events, holding and illegal contact were both off the table. The flag thrown could not be applicable to anything but PI, and you’ve already acknowledged that Brady’s underthrow was uncatchable by Gronk. It’s absolutely a crappy way for a tight, fun game to end, but the ruling could not have been anything else.

  8. Bill White says:

    Hey, don’t leave out hockey and basketball! In those sports things are so critical that the last minute of each period must be timed to the tenth of the second! Which leads me to wonder that if the technology is that good, why not down to the last thousandth of a second? It would make for great journalism, if nothing else: “Carmelo Anthony stole the ball with 4.739 seconds remaining, and his pass to World Peace sealed the win when he sunk the winning shot wit .0216 seconds remaining.”

    Except that Carmelo seldom passes and the Knicks rarely win…

  9. Blake says:

    Anything that makes Bill Belichik unhappy, I’m for it.

  10. Ernie Block says:

    Also being one happy to see the Panthers win, my understanding is that the appropriate call would have been defensive holding. Makes sense.

  11. Jaremy says:

    I think the faux precision that drives me the most nuts is the first downs being “measured to the nanometer” – the ball is downed, often obscured, at which point the ref runs over to make his best guess where to spot the ball, then two guys run across the field to measure the distance with precise accuracy. Give me a break.

  12. mbauer8286 says:

    LOL @ “grumpathon”

  13. SBMcManus says:

    Don’t forget that in soccer the referee has up to 22 extremely helpful advisers on hand at the end of any play – you can often see them circling the referee and providing very reasonable input on what they have just seen, usually by gesticulating wildly with their arms.

  14. Andrew says:

    Joe, you have just made a very good case for the soccer system, as opposed to the gridiron system.

    Both sports involve massive amounts of subjectivity in officiating. Whether something is a hold or a legal block is based on the official’s subjective judgment. Same thing when deciding whether a (soccer) tackle is legal or a foul. There is no bright line separating legal from illegal.

    The difference is, soccer embraces this subjectivity by giving the referee freedom to adjust. Watch any world-class soccer game, and you will see this in action. Penalties are the prototypical example. If an attacking player goes down in the box due to contact from a defender, and the ref swallows his whistle, you can be damn sure that the ref is going to be extra sensitive to any later penalty appeals. The concept of the “make-up call” is familiar to all sports fans; in soccer, it is institutionalized.

    On the other hand, in the NFL, you have an incongruous mix of subjectivity and ruthless objectivity. Best example: precisely measuring first downs based on extremely imprecise estimates of where the ball-carrier went down.

    The soccer system isn’t perfect – it does give an ungodly amount of power to the man in the middle – but it allows for more appropriate “literary” outcomes. Meaning, the team that deserves to win gets the win. Isn’t that all we can reasonably ask for?

  15. DjangoZ says:

    Soccer should have 2 more officials. Just on the opposite side of the pitch from the other linesman. There are too many critical calls missed in soccer.

  16. Chris Smith says:

    The NFL has just gone beyond reason when it comes to getting the calls “right.” I cannot watch it anymore. Every nice catch, touchdown or otherwise, is followed by some review. They sometimes review THE PLACEMENT OF THE BALL at the line of scrimmage. Every review comes with at least a 30 second commercial. There’s just no flow to the game at all anymore.

    I like it when basketball games are called loosely, aka “letting them play.” I like it when there’s a close call at 2nd base on a stolen base attempt. It doesn’t hurt the games at all! (Except for a few instances, like that time the ump called the guy safe at first base ruining that dude’s perfect game.) It’s just entertainment!

    I could understand putting in some sort of stringent replay for the Super Bowl, but the NFL has just gone overboard.

  17. BeninDSM says:

    How did this whole article go by without bringing up the absurdity of the chain gang being the ultimate arbiter up to the 1/2 inch of the first down line because of course they can eyeball a perfectly straight line across field to the sideline and run out for a measurement in a perfectly straight line.

    Also did anyone else imagine Bill Belicheck’s dungeon Lions to be Ndamukong Suh and Nick Fairley?

    • o2yakker says:

      It’s really not eyeballing. When the chains are set on the sideline, a marker is clipped to the to the chain at a “major” yard line, e.g. 5, 10, 15, 20, etc. When the chains are brought onto the field for a measurement, the marker is first positioned on the appropriate yard line before the chains are stretched out.

      • KHAZAD says:

        But when they set up the marker to begin with it is eyeballing. The chains are set up by the guy eyeballing the placement of the ball from the sideline. Then the official on the field eyeballs the forward progress.

        After that, it is measured down to the millimeter.

  18. I listened to John Madden on KCBS radio (also available on the internet; he does about a six minute segment every weekday, with two on Mondays and Fridays during football season). He described one play they had for Cliff Branch (fast Raiders receiver) in the red zone near the 20 yard line where the quarterback was to release the ball when Branch was at the ten yard line for Branch to catch the ball just inside the rear pylons of the end zone. In other words, Branch could cover 20 yards while the ball was in flight. His point was NFL receivers are *FAST* and an unheld Gronkowski might not have caught the ball in front of him where it was intercepted, but he certainly could have been tussling for it. If it wasn’t pass interference, it had to be defensive holding.

    Madden also made the point that even if the penalty is called, it doesn’t guarantee a Patriots touchdown on the next play.

    To me, having now watched the play, I don’t think Gronkowski could have adjusted. He’s a big man running dead away from the badly underthrown ball, However, I note one common thread in so many of these horrible refereeing decisions (this one, replacement refs, the tuck rule, the groundskeeper clearing the snow before the end of game football) the ruling favors the home team. It’s as if our referees are being biased against the visitors because of news reports about dead soccer refs following controversial calls. I don’t think that would happen in America, but I think it’s a consistent enough pattern that there must be some kind of override official on end of game mistakes, somebody far far away and unnamed, to take the heat off of field officials who seem to be showing homerism.

    • Robert says:

      There is some science around the idea that officials favor the home team due to the subconscious impact that the cheering home fans have…ie, a (very good) official is more likely to get a close call right if he’s watching it on a muted monitor at home and more likely to miss it on the actual field of play. Home teams getting (slight) benefit of the whistles is the best explanation we have for the entire notion of home field advantage. There is much doubt that road games impact athlete performance much at all. As one example, free throw shooters aren’t measurably worse on the road with fans waving their arms behind the basket than they are at home with the crowd quieted. (if I’m remembering the data correctly…it’s been a couple years)

      The tuck rule call wasn’t bad officiating, it was a bad rule. They technically got that one right. I believe the Pats/Panthers play was very, very close and wouldn’t protest it going either way too loudly.

  19. Marco says:

    I didn’t watch the Pats/Panthers game last night, but I saw the gif of the Gronk no call here:

    From the GIF, my take is that Gronk would have had a chance to catch the ball if (a) one player had not jumped the route and intercepted it, and (b) he had not been pushed by a defender. Further, I believe that had (b) not occurred, he would not have caught the ball because of (a).

    To the rule book:

    8-5-3-(c): Contact that would normally be considered pass interference, but the pass is clearly uncatchable by the involved players, except as specified in 8-3-2 and 8-5-4 pertaining to blocking downfield by the offense.

    The rule is silent on the reason for the uncatachability. It doesn’t have to be uncatchable because of where it is thrown (which is what I assumed the rule was), just plain uncatchable. In my mind, the other defender jumping the route means that there was no reasonable expectation that Gronk could have caught the pass, so the call for no PI was the correct one.

    • Karyn Ellis says:

      Exactly–dude jumped the route, and Gronk had no chance.

    • Bob Lince says:

      Thanks for the link.

      It appears to me that Gronk doesn’t see the ball until it’s too late. His pass route, before any contact, will not lead him to where the ball is (under-) thrown. If he had seen the ball earlier, he would have made his cut earlier.

      The defender who intercepts does see the ball and has enough time to bump into Gronk, run a few yards, and make the catch. He seems to make his move before Gronk realizes where the ball is going, and attempts to hold up. By then it’s too late.

      IMHO, good “no call”.

  20. Mark Daniel says:

    That was pass interference. No doubt. The refs blew it. Gronkowski made a movement in the direction of the ball when he was standing shoulder to shoulder with the Panthers guy who intercepted the ball. Right at that moment, the other Panther player grabbed Gronkowski and pushed him out of the way.

  21. Cuban X Senators says:

    Soccer ends exactly as basketball or football end.

    Those games don’t end with the clock hitting 0:00 . . . the ball is in the air, the play is live, we wait until the play is dead. A soccer game these days is never called with a team pushing the ball forward . . . the play is live, until possession is taken, the ball goes out, the team in possession plays a long pass backward . . . sure it’s not really in the rules, but that’s what everyone expects . . . and it’s no different than allowing the last heave or the 18 lateral kick return.

    • John says:

      You make an interesting point, though it’s not *exactly* true. A basketball game will end if the player barely fails to get a game-tying shot off before the light comes on. Similarly, if a quarterback rushing to the line for one last shot at the end zone fails to get the snap off in time, it’s over. The equivalent in soccer (the match being called with the a player about to take a shot) basically never happens. But it is true that basketball and football don’t always end *exactly* when the clock hits zero. I guess that makes hockey the strictest in terms of time, because a goal will not count if it crosses the goal line after the clock expires, even if it was already on its way. Even if it’s only *mostly* across the line when the clock hits zero it doesn’t count.

  22. Cuban X Senators says:

    I see no difference between receiving a kick at one’s 25 yard line with the clock expired, lateraling/passing 6 times & scoring in the two sports.

    In fact, I believe, in American football if I’m tied to down 3, I can fair catch with zero on the clock & still have my free-kick to change the outcome – another play after the clock has expired.

  23. KHAZAD says:

    I was rooting for Carolina to win, but it was clearly pass interference. Gronkowski was about one yard away from where the ball landed and turning towards it when Keuchly grabbed him and started driving him back. If Kuechly had time to drive Gronkowski 5 yards back before the ball was intercepted, you can’t really say that Gronkowski couldn’t have moved the yard to make a play on the ball. I am not saying he would have caught it, his chances would probably be less than 50%, but definitely catching it is not the standard, and I believe he would have at the very least gotten a hand on the ball. I have watched him play enough to have seen him make more difficult plays, and I have seen him make catches where it looked like the defender had a better shot at the catch. Without Kuechly’s foul, the ball was definitely catchable.

  24. Chris H. says:

    This last point applied in teh national championship game between Ohio State and Miami, when pass interference *was* called on the final play (or next to final play, I guess). There was considerable focus on whether the OSU receiver was mugged during the catch (contact at that point was pretty incidental – de minimus if you prefer), but the referee explained that he was actually flagging holding while the receiver was running his route. Since the ball was in the air, though, the penalty assessed was pass interference.

    It would have been a lot clearer to call it holding, but of course the penalty for holding is different from the penalty for pass interference.


  25. John H says:

    It’s amazing the lengths that people in this thread and others will go to to pretend it wasn’t textbook DPI. People hate the Patriots that much. The league has already backed off the “uncatchable” nonsense. Agree that Gronk was unlikely to catch the pass but that’s never been the standard for this type of play. Kuechly inarguably interfered with the intended receiver, denying him a chance to make a play on the ball. It’s not complicated. Bad play by Kuechly would have bailed NE out but officials can’t not make that call.

  26. […] read a recent Joe Posnanski essay that emphasizes the Old World and New World differences in time perception and usage. Americans, it […]

Leave a Reply

Your email address will not be published. Required fields are marked *