Dave Kukfa Security engineer etc.

Data, Information, Knowledge, Wisdom

The DIKW model is a conceptual relationship between data, information, knowledge, and wisdom. Several variants of this model exist, which lack consensus on the definition of each term, and the transformations between them.

Russell Ackoff’s 1989 paper From Data to Wisdom is often quoted to define these terms:

Wisdom is located at the top of a hierarchy of types, types of content of the human mind. Descending from wisdom there are understanding, knowledge, information, and, at the bottom, data. Each of these includes the categories that fall below it – for example, there can be no wisdom without knowledge.

Data are symbols that represent properties of objects, events and their environments. They are products of observation. … Information, as noted, is extracted from data by analysis … Data, like metallic ores, are of no value until they are processed into a useable [sic] (i.e. relevant) form. Therefore, the difference between data and information is functional, not structural, but data are usually reduced when they are transformed into information.

Information is contained in descriptions, answers to questions that begin with words such as who, what, where, when, and how many.

Knowledge is know-how, for example, how a system works. It is what makes possible the transformation of information into instructions. It makes control of a system possible. To control a system is to make it work efficiently. … Knowledge can be obtained in two ways: either from transmission by another who has it, by instruction, or by extracting it from experience. In either case, the acquisition of knowledge is learning.

Learning and adaptation may take place by trial and error or systematically by detection of error and its correction. Diagnosis is the identification of the cause of error and prescription is instruction directed at its correction. Systematic learning and adaptation require understanding error, knowing why it was made and how to correct it.

Information, like news, ages relatively rapidly. Knowledge has a longer lifespan, although inevitably it too becomes obsolete. Understanding has an aura of permanence around it. Wisdom, unless lost, is permanent …

… information, knowledge and understanding all focus on efficiency. Wisdom adds value, which requires the mental function we call judgement. Evaluations of efficiency all are based on a logic which, in principle, can be specified, and therefore can be programmed and automated. These principles are general and impersonal. We can speak of the efficiency of an act independent of the actor. Not so for judgment. The value of an act is never independent of the actor, and seldom is the same for two actors even when they act in the same way. Efficiency is inferrable [sic] from appropriate grounds; ethical and aesthetic values are not. They are unique and personal.

The DIKW model is summarized in the following flowchart (from Wikipedia):

DIKW flowchart

[T & E meaning tacit and explicit]

Gene Bellinger et al. present an alternative view:

Personally I contend that the sequence is a bit less involved than described by Ackoff. The following diagram represents the transitions from data, to information, to knowledge, and finally to wisdom, and it is understanding that support [sic] the transition from each stage to the next. Understanding is not a separate level of its own.

DIKW graph

The DIKW model has its limitations; it isn’t universally applicable, and can break down when analyzing the semantic definitions of each step (e.g. drawing the line between data and information). Though I find the model is still a useful concept, which illustrates:

  • the “graduation” of data, information, and knowledge (however you want to define them) into increasingly simplified but useful forms
  • the relationship between stages of the DIKW pyramid and timelessness
  • the difference between operational efficiency (doing things right), and directional effectiveness (doing the right things)
  • doing the right thing requires knowing how the system currently works
  • how the value of an act differs between individuals

Navigating Complexity

The complexity and nuance across the interconnected domains of knowledge in the world make it challenging to progress on social issues. For example, an environmentalist’s goals are connected with environmental science, engineering, manufacturing, economics, and politics, among others. How well should one try to understand these interconnected domains to progress on their goals?

The general consensus is it depends on one’s strengths, which is echoed by:

Scott Page

From The Knowledge Project:

One of the things I talk about in both The Diversity Bonus and also in The Model Thinker is that you can think of yourself as this toolbox and you’ve got some capacity to accumulate tools, mental models, ways of thinking. What you could decide to do is you could decide to go really deep. You could be the world’s expert, or one of the world’s experts, on random forest models or goals or the Lyapunov functions. You could be one of the world’s leading practitioners of signaling models in economics. Alternatively, what you could do is you could go deep on a handful of models, where there could be three or four things you’re pretty good at. Or you could be someone who I think … a lot of people are really successful … by having just an awareness of a whole bunch of models. Having 20 models that you have at your disposal that you can think about. Then when you realize this one may be important, then you dig a little bit deeper.

The world is a complex place. I think that the challenge is to become a more nimble thinker, is to be able to move across these models. But at the same time, if you can’t, if that’s just not your style, that doesn’t mean there’s no place for you in the modern economy. To the contrary, it means that maybe you should be one of those people who goes deep.

The point of the core philosophy of The Model Thinker is even if you do the best you can, even if you’re a lifelong learner, even if you’re constantly amassing models, you’re still not going to be up to the task of solving any one … you yourself are not going to solve the obesity epidemic. You yourself are not going to create world peace. You yourself are not going to solve climate issues. Your brain just isn’t going to be big enough. But collections of people by having different ensembles of models, creating a larger ensemble of models actually have a hope of addressing these problems.

You need this weird balance of specialist, super-generalist, quasi-specialist, generalist. There’s even people who I’ve heard describe … that their human capital is in the shape of a T, in the sense that there’s a whole bunch of things they know a decent amount about and then one thing they know deep. Where other people describe themselves as a symbol for pi where there’s two things they know pretty deep, not as deep as the T person, and then a range of things that connect those two areas of knowledge, and then a little bit out to each side. I think that it’s worth having a discussion with yourself … is to think okay, what are my capacities? Am I someone who is able to learn things really, really deeply? Am I able to learn a lot of stuff? Then think about a strategy for what sort of human capital you develop.

Because I think you can’t make a difference in the world, you can’t go out there and do good, you can’t take this knowledge and this wisdom and make the world a better place unless you’ve acquired a set of useful tools, not only individually, but also they’ve got to be collectively useful. Because you could learn 15 different models that are disconnected, that apply to different cases, and never have any sort of gestalt, any sort of whole, and that might make it hard for you to make a contribution. Or you could say, I’m going to be someone who learns 30 different models. But if you’re not someone who is nimble and able to move across them, that may be more frustrating for you.

In a complex world, your ability to succeed is going to depend on you filling a niche that’s valuable, which as in Barabasi’s book, it could be connecting things, it could be pulling resources and ideas from different places, but it’s going to be filling a niche and that niche could take all sorts of different forms.

Atul Gawande

From The Knowledge Project:

I majored in biology, but I also majored in political science, kind of looking for… there must be more to the world than just medicine. And I found it. I found it in lots and lots of different places. Some in science; I worked in a lab. Some—you know, I tried everything in college. I was in a band, I learned to play guitar, I wrote music reviews for the student newspaper. I joined Amnesty International. I worked on Gary Hart’s very shortlived campaign for president as a volunteer. Then, when I got out of Stanford I went on to do a master’s degree in politics and philosophy of economics at Oxford, out of hope that I could maybe do a graduate degree in political theory or something like that. I just found out I wasn’t very good at those questions and a lot of the things that I tried I just wasn’t really made for or cut out for. And I kept coming back to medicine as a place where I was familiar, I was comfortable. It wasn’t for the best reasons, right? It was a place that I knew and I could thrive.

What I also liked about it was, you didn’t actually have to decide what you wanted to be when you grew up. It deferred all kinds of decisions while I figured out everything else along the way. So when I got out of graduate school and decided to just stop with a master’s degree in philosophy… then I worked actually in politics for a couple of years on the Hill and found I didn’t want to just work in politics.

I kept finding myself gravitating back to medicine where you could have skill… the values were at the core of it for me, that it was about grappling with how science meets humanity in a place where—and policy and the world and all the complexities of life—in a place where you could really think about the individual in front of you, but also the system as a whole, and I wanted to somehow connect on both levels.

I like having a lot of irons in the fire. I like being a jack of all trades. Finding the edges between things is often where I have something to add. You know, if you look at what I contribute in these spaces, it’s not genius ideas. A checklist for surgery, it’s just taking an idea from one domain and saying let’s bring it over to the other and see if it can work, or understanding what people’s goals are when they face mortality and end of life. A lot of them just come from digging in deep enough to understand the gap between what we’re aspiring for and the reality of what we’re doing, and then trying to figure out where the bridge is to a narrow that wide gap.

I think I grew up kind of interested in how the world worked, and I had a very limited vantage point in my town in Ohio growing up. And every opportunity to see more, my handle hold, was through science. My parents were doctors and that gave me a way of seeing and thinking about the world, but then my parents were also people who were deeply involved in the community and trying to deal with the challenges in a community that had a college, but was also the poorest county in Ohio. My brain worked in such a way that, I loved understanding the ideas at an ideas level and then trying to figure out how you ground it. So I was always looking for ways to understand the world, and that meant needing to bridge and look more widely. And so each move, college and then going beyond, kept widening that, and I’ve just loved that. I’ve loved adding another space that I could explore and it was only by happenstance, it was very late that I found I had anything to contribute. That really wasn’t until my thirties when I finally found I could connect the dots between different things I had been learning about.

Charlie Munger

From the 2016 Daily Journal Annual General Meeting, on whether he is in favor of specialization or taking a synthesis/multi-disciplinary approach:

Saying one is in favor of synthesis is like saying one is in favor of reality. It is easy to say we want to be good at it, but the rewards system pays for extreme specialization. You are usually way better off being a deep expert [in one thing] than someone an inch deep in a lot of disciplines. It [synthesis] is helpful to some but not the best career advice for most people. The trouble is you make terrible mistakes everywhere else without it, so synthesis should be a second attack on the world after specialization. It is defensive, and it helps one to not be blindsided by the rest of world.

From the 2017 Daily Journal Annual General Meeting:

I don’t think operating over many disciplines as I do is a good idea for most people. I think it’s fun, that’s why I’ve done it. I’m better at it than most people would be. And I don’t think I’m good at being the very best for handling differential equations. So it’s in a wonderful path for me, but I think the correct path for everybody else is to specialize and get very good at something that society rewards and get very efficient at doing it. But even if you do that, I think you should spend 10 or 20% of your time into trying to know all the big ideas in all the other disciplines. Otherwise… I use the same phrase over and over again… otherwise you’re like a one legged man in an ass-kicking contest. It’s just not going to work very well. You have to know the big ideas in all the disciplines to be safe if you have a life lived outside a cave. But no, I think you don’t want to neglect your business as a dentist to think great thoughts about Proust.

Tyler Cowen

From The Knowledge Project, on Munger’s thoughts on specialization from the 2017 Daily Journal Annual General Meeting:

I mean maybe most people, but you know it’s person by person and for some people it should be 50–50. Certainly at the higher levels I think generalists are important. If you look at CEOs several decades ago, most CEOs were people hired from within that sector and now a CEO is much more often hired across sectors. So someone who, you know, worked for an oil company would then be hired to run a manufacturing firm. So that’s showing some kinds of knowledge are actually more general.

If you think, well, insiders have some kind of natural advantage in having, you know, an inside track, if companies are more willing to hire these outsiders, I think that’s a clear sign that executive knowledge is becoming more general in nature, more global, more a set of skills about communicating, understanding how politics, global economy, internal management all tie together. Those are somewhat general skills. You do need to understand something about your sector too, though.

Theories of History

Explained in both essay and video formats by Samo Burja:

Everyone has an implicit theory of history– usually inconsistent across time periods and typically incoherent without explication and conscious work, it will nonetheless be the basis of much of your action in the world. Most people never discover theirs simply because they don’t realize they’re acting on one. Now that you have the concept– what is yours?

Institutionalization of Social Movements

From Interest Groups and Lobbying:

If a social movement succeeds in cracking open the political system, it must then learn to play by the rules of the institution it has worked to crack. Movements rearrange some of the pieces of the power structure that supported policies at the root of their grievance, forcing back older, more entrenched interests by making them accept the legitimacy of the movement’s demands. They add their interest to the array of interests deemed worthy of government support, expanding the range of policy problems receiving government redress. This success is significant, but victory comes with costs, which may not be immediately apparent to the organizers of the social movement, who are now looking at careers as lobbyists. They must play by the rules and norms of the game they have struggled to join. The movement changed the beneficiaries of public policy and the values it enshrines, but not the way it is made. Activists-turned-lobbyists are now expected to work with lawmakers through the regular, institutionalized procedures of the political system. Bills will be introduced, other members of Congress will be lobbied, information packets developed, email campaigns launched—all with a united message that benefits their new legislative allies as much as their members. Other interest groups must be recruited as allies or bargained with as opponents. Because operating on the inside is much cheaper than staging protests, the groups staff are usually happy to oblige.

Passionate, dedicated movement members may not understand this transformation. They may not understand that further pursuing their interests now requires compromises with competing interests and paying attention to the needs of lawmakers. They may not understand the growing gulf between themselves and their leaders as the latter become part of the Washington social scene. While in many cases this simply leads to the loss of dispirited, disillusioned members, political scientist Anne Costain (1981) describes a much more dramatic result. At the 1975 national convention of the National Organization of Women (NOW), members forced out much of NOW’s Washington staff for the crime of playing by Washington’s rules, even though the professional activists had learned how to use them to the members advantage.

Social movements are therefore embryonic interest groups, though not all interest groups start as social movements. The trade and professional associations representing businesses and white-collar professions certainly did not start out as street protests. Citizen groups, on the other hand, often started out as some kind of social movement, their origins grounded in passionate political activism. Sometimes they still try to keep the trappings of it and fire up large numbers of members just to keep them involved in the group. Of course the real difference between the two is that social movements represent interests truly excluded from regular politics but who have the opportunity and resources to break in, whereas interest groups are already on the inside.

Revolving Door in Politics

Simple definition from Wikipedia:

Governments hire industry professionals for their private sector experience, their influence within corporations that the government is attempting to regulate or do business with, and in order to gain political support (donations and endorsements) from private firms.

Industry, in turn, hires people out of government positions to gain personal access to government officials, seek favorable legislation/regulation and government contracts in exchange for high-paying employment offers, and get inside information on what is going on in government.

The lobbying industry is especially affected by the revolving door concept, as the main asset for a lobbyist is contacts with and influence on government officials. This industrial climate is attractive for ex-government officials. It can also mean substantial monetary rewards for the lobbying firms and government projects and contracts in the hundreds of millions for those they represent.

From Interest Groups and Lobbying:

As noted above, 80 percent of lobbyists came into lobbying from government jobs. Lobbyists told Heinz and his colleagues that government service helped launch their lobbying careers. Of the lobbyists interviewed, 70 percent said it gave them familiarity with issues, 80 percent said it taught them how the lawmaking process works, 59 percent said it gave them important contacts in Congress, 48 percent said it gave them contacts in the administration, and 47 percent said it helped them gain contacts with other lobbyists (Heinz et al. 1993, 115).

This is what is colloquially called the revolving door. Many ex-Hill people end up at well-known lobbying firms such as the Podesta Group (which has the most ex-Hill staff at eighteen, according to Washington Representatives 2010, vii) and major interest groups such as the US Chamber of Commerce.

Why? Tony Podesta, founder of the Podesta Group, told the Washington Post that “people who are experienced in Washington tend to be better at doing this kind of work than people who have never worked in the government before.” Often the reason is strategic. The Motion Picture Association of America hired former senator Chris Dodd (D-CT) to be its new president because for years Dodd had chaired the Senate committee that controlled many of the policies important to the group (Romm 2011). Conflicts of interest? Maybe. Citizens for Responsibility and Ethics in Washington found in 2011 that seven ex-congressmen were now lobbying for interests these legislators had supported with government appropriations while in Congress (Farnam 2012b).

The hiring of ex-legislators and legislative staffers as lobbyists (and the issue of who gets hired) has become a political issue. Interest groups and lobbying firms try to staff up with lobbyists of the same ideological persuasion as the party controlling Congress. For example, Republican staff members were in high demand in 1994 after their party gained control of the House and Senate (Stone 1996). Party leaders even threatened to shut several interest groups out of Congress if they did not hire Republican staff as lobbyists.

In the past, well-connected lobbyists … could easily work with both parties, bringing competing groups of interests together to hammer out deals and resolve conflicts (Ignatius 2000). Today, though, lobbyists are pressured by the parties to take sides, with their access threatened (which kills a lobbyist’s career) if they do not.

Perhaps even more interesting is that 605 congressional staffers in 2011 used to be lobbyists. When Republicans took over the House of Representatives that year, many new legislators recruited their senior staff from among the ranks of major lobbying firms and business associations (Farnam 2011). The door fully revolves. Government workers leave to make money as lobbyists, and some later return to work on the inside, possibly still biased toward the interest groups that recently employed them, and they could return to lobbying again in a couple of years for a lot more money.