Individual assessment: Reflection, questions and research

Speaker A So individual assessments, we'll start with calibration sessions and I don't have much to say beyond what you all have said. I think that I use the methodology that most of you use where we plot roles by level and then look at people relative to the roles. I do have one question, maybe a few more that I wrote over there relative to that and I brought it up at the beginning of the week and I'm still curious about how many of you, when you level roles, use the description of the nature of work versus time span or a combination or some other thing. And part of the reason why I'm curious about that and we don't have to answer all that now, but these are my questions, so I'm bringing them would be how does this influence trust and fairness in the organization? If time span is a measure, whereas a description of the work is a judgment and judgments can be subject to bias or prejudice or distortion, then it seems to me personally that you would want to lean toward time span if you could because it's actually a measure. However, I recognize that many managers do not like this methodology and it can get you into more trouble than it's worth sometimes. So we generally use both. But I'd be curious to know your methodologies and Ken, you can tell me where it might be appropriate to actually have that discussion if we do. And then another question that falls on that is functional snobbery. Curious to know from this process when you calibrate across the organization. We were working in an engineering organization at one point and they were absolutely appalled and shocked to hear that there would be human resource work at level three. Are you kidding me? We are engineers. You have some of our roles at two and there is human resource work at three. Are you kidding me? So I'm just curious about your experiences with functional. So I call that functional snobbery and I find it in various places. I ask that also because then we can go back to if we're using time span then we can to a certain extent defend the reason why there are HR roles at three. Not as straightforward using descriptions of the nature of work, although you can use that as well. And then that then goes back to this whole issue of trust and fairness. How do you keep people from feeling like you are gaming the system? That would be my question and then this last one is one more question around this methodology is sometimes where I run into time span issues is we will have what is clearly a level two role by the nature of work embedded in it. So in other words, it clearly involves cumulative analysis, diagnostic type work where I have to add things together, make a hypothesis and decide how I'm going to proceed. It's not a fully prescribed output like you would get it for. So we can all agree based on the nature of work, this is level two work, or it requires level two capability. And yet there are a few instances where we can't seem to find a time span beyond three months. Then my question becomes an example was even I think we had a commissioning engineer who goes in and does commissioning and is out within three months and then goes on and does the next commissioning and is out. And there's nothing else expected of the employee. Another one would be a loan officer in a financial institution. They're making loans after some digging and we found out they are not just saying go no go, it matches and we'll approve it, or it doesn't match and we won't approve it. There is actually some deal making, if you will. So they're expected to use their judgment to say, the way you delivered me this paperwork, it won't work. But if you move this or you add that or you take the husband off the application, we can make it happen, which is level two work as opposed to level one. And yet again, we're struggling to get time spans beyond three months. So I'm just curious. Then my question becomes, my thinking is in independent contributor roles at level two. So non managerial roles at level two, because often level two roles in a managerial sense, the longest task is the training of new employees. So when you take that away and you have an independent contributor, they don't have that longest task. So then my question comes back to is the longer tasks at l Two within a hierarchy improvement work? So in other words, it's more project oriented work. So my day to day work is approving loans. And yet in a managerial hierarchy, I might have some project work toward the continuous development of the department. So that would be designed at three. We've got a continuous improvement project. I get a piece of it at four. That is a four or six month project piece of work. And the particular organizations where we're finding these l Two roles that are clearly level two by nature of work and yet don't throw up the time spans, we would expect the level three work isn't being done. So what that's saying? This is my question. My hypothesis is, is that because then if the l Three work is not being done, the l Three projects, continuous improvement projects are not being done, and that project work at the four and six month piece is not being given to the level two. And their longest task would be the gift back to the organization in the continuous improvement bucket. So then I would ask the organization, if you are not using these people at the longer time spans, are you getting your money out of the person? So those are all my questions.

Speaker B Let's do that in discussion after you finish.

Speaker A Okay. So those are the things I'm puzzling with here. Barry and Sheila just took care of this one. When I have done talent pool gearing in an organization, we plot CPC only, and we tell the managers, although CAC is absolutely important and valuable and useful in performance management and other places, right now we're just looking at CPC, put the other side. And what I find quite often is that when you have someone, let's say, who's capable at level four CPC, and yet they're in a level three role and they're not delivering, it just kills the manager to rate them at four. They don't want to do it. And you have to kind of know, okay, I get it. I realize they're not delivering. And I'm not saying we would promote them. I'm not saying any of that. I'm just asking you, if they were working at their full potential, where could they be? So we've had this struggle internally around. Should we go ahead and plot both at the same time to take care of that anxiety so they can say, okay, yes, they're here, but they're only working here, and so you've solved that problem for me. You're doing that anyway, and thank you for that. So that's my thoughts on assessment during calibration sessions. Nothing more to add about process than beyond what you all have said. Assessment of incumbents. We would do this very rarely. Haven't done much of it. We have done it, and it's usually for an internal if someone wants to change from one department in an organization to another, and the hiring manager is requesting the outside opinion, if you will, judgment. And what's happening is what we recognize is, yes, in a perfect world, we would have the managerial judgments, and we would use that. However, if your organization that you're working with is not yet requisite and so in one example, we have a client where it's a level five organization and one particular function has seven levels. It's quite a mess, has been for a while, and they're being quite patient, but the people are trying to get out of that department and into others. However, the other managers are suspect about the accuracy of the judgments being made in that type of environment. So in that sense, they're using it as a triangulation point between what's being said, what's their experience, what does their interview say? So that would be the circumstance. Generally, where we do that, it's not a common practice. We don't generally do it. Okay. Assessments for recruiting, which is what I've been spending a lot of time doing lately. I was trying to tally in my head about how many over the course of my working in the requisite arena, I think I've probably done. I'm guessing between seven and 800. And about 500 of them have been in the last two and a half years for a particular client. So what are we doing there? Again, forgive me, I don't know how familiar you are with this or not, but essentially what we're doing is we're looking at we are engaging someone in an argument. We're asking them to take a position, solve a problem. We're looking at the structure of their argument, and then we're making a determination about the structure of their argument, which would start at level one with a declarative argument. Level two would be cumulative, level three, serial, then parallel, and then it repeats. So level five, we're back to declarative level six, cumulative. And what would change then would be the order of information the person is using. Okay. As I'm making these judgments, I am fully aware of the fact that I am also informed by their resume. I use that to form an initial hypothesis about where they may be based on the work they're saying they've done or have done. I am influenced by the content. So if they are talking to me about considering outsourcing and the trade offs associated in the hit their brand might take, I'm thinking, okay, they're not at two, they're probably not at three. We're probably up a little higher. So I am influenced by content. And then I also just lately, having done so many of these in a short period of time, am starting to although I cannot articulate what it is, I'm starting to sense mode. The one particular client that I've done the 500 interviews for, they have a sweet spot for finding mode six people. Most of them are at mode four currently, but they're mode six. And so now, having done so many, I can get a sense of up. This is higher than six, which is unusual, or this is not six. This person's not traveling with that pack. And so if I get to a place where I'm saying to myself, is this too high or is this three low, I will go to the progression chart and if it makes a difference, too high puts them at mode four, and three low puts them at mode five. I'm starting to use that as a help to say he feels more like a mode five. So I'm going with a higher rating. Research Needs I am usually again, working as a wholesale consultant under the umbrella of another consultant. So I'm not there during the contracting phase with the client. One of the things I would love to see is the clients who use us for this type of work, if we could gather data and then go back and see who they hired and what the judgments are about their level of capability from the internal managers, and align that with our judgments. Pre hire and see how they align. Haven't been able to get a client who's interested in doing that yet, but would love to. We're pushing one of our clients to get there. I would love to do this. So learnings in this arena, the things that I've been learning, this is what I'm most excited about and I could talk forever about. So I will watch your eyes and when you start to flutter a little bit, I'll stop talking. How about that? What would I like to say to Elliot Jax, if he were here? I want to know what makes for a serial connection. And the reason I say that is because when I go back and look at the book, Human Capability, the things he credits as serial are pretty light. I don't know that I would credit them as serial in the way that I do this. Some people say cause and effect. Other places in his literature, it doesn't say cause and effect. It says that A sets the conditions for B. So A doesn't necessarily cause B, but A sets the conditions for B. So that's a little bit different definition. So I'd like to have a conversation about that. Excellent. I'm going backwards by one slide just because I added something to a later version of this slideshow that's on my stick but not on my PC. The best anecdotal evidence we have around accuracy of pre hire judgments versus on the job performance is, and this is from a few years back that my ratings and Glenn Meltred or my colleague tend to be on the higher side than what they feel like they're experiencing when the person gets into the organization. So the question that throws up know, obviously is it CPC or CAC that they're feeling? Obviously CPC is going to be the highest and CAC would always be lower, but is there something more to that? And then that leads me back to this comment around serial connections. Again, I have personally tightened up a little bit in what I'm allowing as a serial connection. And I have been what I might call less generous than I have been in the past. So that could be evidence that I was shooting a little higher. And I have tightened up in that arena just through my own personal experience. My next question to Elliot would be, why did you name that fourth structure parallel processing? And the reason I say this is, again, it was another when you're processing at three, you're using serial pathways, and at high three, you may have more than one serial pathway, right? You might have multiple projects or multiple lines of equipment that are running, but they do not intersect. You do not integrate them. They are not comprehensive. I deal with them independently. So then why is it that at four we now have multiple serial pathways, but they're integrating, they're touching. Well, that's not parallel, right? Parallel lines never touch. So for a long time I had that word stuck in my head and when I would get lines of serial data and I saw more than one, it would lead me to think, oh, this is four, I've got parallel serial pass here. When in reality that's actually high three. That's not four. So for a while, just the way that language was threw me. So I think I would have changed that to something else. Integrative processing, interconnected processing, I don't know, but I don't like the word parallel, so I would ask Elliot about that. I don't know what you'd have to say personal. I've come up with some ideas around what I call mode glare and mode drag. When I first got into this particular organization that was interviewing all the mode six, and these people would blow me away. They're 30 and they're talking about things that I still today don't even understand or think about. It's incredible to me how bright these young people are and the things they're saying. And so I think for a while, again, I was being influenced by the content of what they were saying and there was a little bit of what I call mode glare. Wow. And I think I was shooting a little high. So again, I've recognized the error of my ways and dialed that back a little bit. And now I'm actually experiencing mode drag. And what I'm realizing is when we're working for another organization where most of the people are mode three and four that I'm interviewing, I get this mode drag. Oh, no way. And then I have to slap myself again and say, no, this is level three, but it's mode three and it feels completely different than level three, mode seven or mode six. So these are my own terms, feel free to work them into your literature if you care to. Gotten better. At my questioning, I was talking with somebody at dinner around I'd love to ask this question now, around highest and best use of the particular role that the person is in. So I was interviewing someone for controller and I'll say different organizations might use the controller role in a different way. But if I were to ask you what is the highest and best use of a controller role, what would you say it is? And people will absolutely telegraph where they are. It's really incredible. And this is now we're back to the billis and Row bottom description of the nature of work where they'll talk about. I had one person answer that question by saying the highest and best use of the controller role is supervisory. And I was like interesting. And when I was all done, she came out at level two. So the role was about supervising the people who were inputting the data and making sure it was accurate. And that was the controller role. It wasn't about processes, it wasn't about operational effectiveness or the things you might hear at three. It was really quite interesting. So that's a great question. Stratum three. Elliot used to get a little bit upset about leading the witness. And again, I'm discovering leading the witness doesn't make a darn bit of difference. You can flat out ask people for things and if they're not there, they won't give it to you. So when I'm testing for three, I'll ask them for the sequence. A lot of times I'll say project plan it. So if I asked you to create a nonprofit to tackle this particular issue, you were just talking about homelessness in your city, what would your project plan look like? There got to be pieces and parts, how do they roll? How would you sequence those? What do you see as happening first and how would it flow? And if they're not at three, you'll get the and and you usually get A to B and then and you don't get the C. So that's an interesting one at four. Some of the themes and things I like to ask is comprehensive. I ask them about what would a comprehensive talent management system look like and of course everybody can answer that but at three you don't get comprehensive, you get independent serial pathways. Well, it has this and it has this you don't get comprehensive. I like to ask about trade offs as well. I'll ask people about outsourcing if they hit, I look on their resume. If they have some experience with outsourcing, I'll say, well, generally when we make a business decision, it's not all positive. You give up things to gain things. Can you tell me about the trade offs associated with outsourcing? And again, everyone will say, oh well, absolutely, there are trade offs but you won't hear the true recognition of the trade offs in the balancing unless they're at four. So it's quite interesting. One of the things that you don't get, again at three, if you think someone's at three and you're testing for four, you can ask them about reconciling. So you will say to them, okay, you just argued this particular line of reasoning. What would people who are going to argue against you, what would they have to say about this same issue? And usually at three they can tell you and then you ask the question how would you reconcile that? Great question and at three they will tell you well, I know they say this but it's rubbish and they completely throw out the opposing argument or they'll tell you why I want data, I don't really believe it. They cannot hold in their mind the opposing argument and balance it. So we're back to that trade off thing again is they will completely dismiss the opposing argument if they're at three. At four you get well, absolutely. If we regulate the financial sector, it is going to create a burden on the banks. There is a cost issue there. However, we absolutely have to do it because we've got to make sure that we have fair points of entry for so there's the back and forth. Absolutely this and so what you'll hear at four is we need to do it, but only to the extent that so there's a portfolio view at four that you don't see anywhere else. You don't see at three, two, one. At three, two, one, it's either or it's good or bad. We need it or we don't need it. Whereas at four, it's, well, we need this, but only at this ratio. And this needs to be 80%, this needs to be 20%. And there's this ratio portfolio type thing going on in the mind of someone at four that you don't see earlier. And then finally, the whole idea around naming things at Stratum five, we talk about people speaking conceptually at five, and I'm convinced it's a naming thing, but I know more about what that means, and so I'll keep going. I've learned a lot about this by interviewing my children. I have them argue for me, and I tape record it and I type it and I study it again. We know that we have declarative, cumulative serial and parallel, and at five we get declarative. But also we know that sub one, we have parallel. We have a set down there as well. And so my children have been down here, and I've used that to learn, if we go here from a lot of talking and all my serial pathways and my integration and all these words that happen at four, a lot of mouth diarrhea, because people are having to explain every beast to you. And then up at five, suddenly they become more quiet because they can put it all into a package and name it, they become quiet. Well, what does that look like? Then? You would expect the same thing here. Yes. So this is what I've learned from my children, is that my daughter a couple of years ago, had to write a book. She wrote a book on dolphins, and she had to have chapters. Of course, each chapter was one page, a sentence and a picture, but that's what it was. And these were her chapter titles what a Dolphin Eats, Where a Dolphin Lives and what a Dolphin Looks Like. So at age, I don't know, she was seven or eight, these were her chapter titles. So very descriptive, right? But we weren't yet at third order, where we actually have a category name. And so what I would propose you would hear if someone were writing that same book and they were at Level One, we would hear something along the lines of dolphin diet, dolphin habitat, dolphin anatomy. Right? So these are the Level One type words you would hear here. She was describing these are more categories, and we could have monkey diet, we could have dog diet. Right. So there's some portability in this idea. Well, here she was talking about what a dolphin eats. So I was trying to pick around and say, what can I learn from this that might help me here? And what I'm discovering is when people I think people are at four and I ask them questions and they spend a lot of time building a complex model for me. I'll ask them about immigration policy in the US or change management. I'll ask them something. They'll build this very complex model. And then I will say to them, let's pretend like you were writing all of that on a flip chart. So you've told me a lot of things. You've built a model for me. Now I'm going to come up and draw a line across the top of the flip chart and I want you to name that framework or name that model that captures the essence of the model. Can you do that for me? And these are the kinds of things that I get that are not five because they are descriptions. Translating strategy into action process. It's like what a dolphin eats. A realist view to immigration or insightful change isn't that difficult. So it's not five, it's not a concept. Yet it's embedded in this thing that you just asked me about immigration and I couldn't take that model and use it as a machine or a framework anywhere else in the world. So then these are actual examples that I got from people. And then I just said, what would this maybe look like at five? And they don't have to be eloquent or something you've never heard before. But translating strategy into action process at five would sound something like it's a strategy translation model. But it's a thing. It's an it, it's an entity here that I can look at. It has form. Whereas here I'm just making a description, a realist view to immigration. I said irrelevant immigration policy. Somebody might have been arguing that back when the US was set up, we were taking in anybody we have. Our Statue of Liberty says, give me your poor. But now times have changed. We need to think about jobs and who we need. So we need an error. Relevant immigration policy would be something you'd hear at five that you wouldn't hear four insightful changes isn't difficult. You might get something like a gentle persuasion change model. But it's a thing, it's a name, this is a description. And it's becoming much clearer for me. Now. Of course, again, anybody can say these things. You don't have to be at five to actually say these things. So to me, what I'm looking for is self generation. I'm looking for things that they have put a name on. So I don't know if that's an appropriate test or not, but I've been using it. Let's see. Okay, I already talked on this about some of the things I learned about serial in parallel. So that's it.

Speaker B I want to run this idea by this group because I've thought about this quite a while. When you look at the different levels of thinking and I trying to make sense out of it, being an engineer, and I'll start from the top. The top level I call systems thinking that the person is able to see whatever problem is it's a system. And in engineering terms, those are taguchi experiments. Are you familiar with taguchi? All right, so you might have eight variables. You may have eight settings in the variables. You can run a series of experiments of maybe eight, and that'll give you an indication of which of the two or three variables are driving the system. And those are the ones that you want to focus on. So people are really smart do this in their head, and they know what to ignore, and they can bore in and those things that are really driving the system. So that's what I see as the highest level of thinking. The next one down is the cause and effect. This causes this and that's the ishikawa Diagram Man method, materials, money and materials. Right. And so you can see put those all together, and you analyze it, whether it's giving you a problem or how you correct a problem, but it's a lower level of thinking. And then you get down to the next one. Then you have the Gold Rat cause and Effect tree, which I find is very effective. And that is are you familiar with that?

Speaker A No.

Speaker B Okay. Well, the Cause and effect tree says, well, what causes this? And a lot of times, especially in public policy, they will focus in on high school dropouts, and they'll come in. But if you do a Cause and effect tree, which I do in practice, where you will have people say, well, write down everything that's wrong that you can think of, I don't care what. And then you get the little pieces of paper, and then you start, well, is this a cause or is that an effect? And you build a tree, and then after you're done, you say anything above the bottom row isn't a root cause. You'll see a lot of policy being addressing those things that are up, and if you take them away, it doesn't affect the system. You haven't gone to root cause, and then the lowest one is the declarative. And I always like to say, well, it's like the guys I know in the form, well, I drive an F 150, and that's the best truck. Okay. And if you want to be caught, then you can go the next higher, maybe a little bit. Well, it might give me some reasons, but it's pretty much cut and dried thinking. So I was curious what this especially if you're dealing with engineers. I'm trying to explain what these different levels are. They might relate to it better, that's all.

Speaker A Sounds like you're onto something.

Speaker B Go ahead. Mike here's, Mike.

Speaker C So, Michelle, it seems that you have a lot of experience in interviewing for capability. So when you are interviewing mode six or seven, do you think there's a correlation between what your level of capability needs to be in order to assess high level of capability?

Speaker A I believe there is. Do I know a specific answer? I'd say. No, I would think there's probably a minimum mode that's necessary. Don't know what it is. And I would say you could probably judge one level higher. When you get two levels higher, it starts to get fuzzy. You know it's up there, but I'm not exactly sure where that's been. My personal experience. Don, you've done a lot of this work as well?

Speaker B I would agree with that.

Speaker A Yeah.

Speaker B BIOS, when they hire people to work with them, generally say we won't hire anybody who's not mode five.

Speaker D Wish would save you what I would like to make a little between brackets. Geometry moment only. In the Greek geometry, parallel never touches because there are other geometries where from a line you can put any quantity of parallel lines from an outside point and in another you cannot find even one parallel line. So things are different in the real world than what the Greeks thought for the plane. Close brackets.

Speaker A You ram.