Toronto Star – After a girl committed 71 violent incidents in 10 months, children’s aid and this caregiver agreed the group home program wasn’t working – 2017-12-01

After a girl committed 71 violent incidents in 10 months, children’s aid and this caregiver agreed the group home program wasn’t working
After a girl committed 71 violent incidents in 10 months, children’s aid and this caregiver agreed the group home program wasn’t working

A troubled teen in a Lindsay-area group home ultimately got the help she needed — in spite of the obstacles. Why did it take so long?

When care worker Stephanie Stokes formed an instant connection with a troubled teen, things began to change. "Aggression went from all the time to pretty much nothing,” says Stokes.

When care worker Stephanie Stokes formed an instant connection with a troubled teen, things began to change. “Aggression went from all the time to pretty much nothing,” says Stokes. LUCAS OLENIUK

She was a bright and kind-hearted girl, haunted by the night.

She arrived at a Lindsay-area group home on Quaker Rd. after moving through several foster homes and suffering an adoption breakdown. Quaker Rd.’s caregivers adored her.

But she dreaded sleep and the recurring nightmare of an intruder breaking into the home and attacking her in bed. At bedtime she would lash out violently, sometimes for hours.

When she injured staff, she would apologize and break down in tears.

Her sometimes epic outbursts — fuelled by symptoms of fetal alcohol syndrome — were described in reports sent to the children’s ministry whenever a serious incident occurs in Ontario involving a child in care.

Through a freedom-of-information request, the Star obtained all of the reports filed by the Quaker Rd. group home in 2015. There were 122 serious incidents that year, mostly violent. The kind-hearted girl was responsible for more than half the incidents.

Her troubled time at the group home highlights the inaction of Ontario’s child protection system, which failed to intervene even as the girl violently lashed out almost twice a week for 10 months. It only moved after the girl was detained by police for assaulting and injuring two of her caregivers.

Her plight reflects a system with lax ministry oversight and no way of knowing if minimal standards of care are being met. There is also no way of tracking if vulnerable, high-needs kids are getting the care and treatment they require.

The girl’s anguish came to a head on Dec. 15, 2015.

Around 10 p.m., as a caregiver was rubbing her back to help bring on sleep, the teen became agitated and started screaming for another resident.

The caregiver tried to hold her in a “nurturing hug,” but she kicked and pushed, according to the report to the Ministry of Children and Youth Services.

It was the beginning of an all-night rampage. She ran into the bedrooms of the home’s five other residents, screaming for them to wake up. She punched and kicked the walls, tried to pull the fire alarm, jumped on the dining room table and tore off ceiling tiles. She head-butted and repeatedly punched two caregivers, and kicked one of them in the groin.

The mayhem finally ended at 6:15 the next morning after staff tackled the girl to the floor and called police. She was arrested and released two days later after a bail hearing. Her caregivers were treated by paramedics.

It was the last straw.

After 71 violent incidents over a 10-month period — each resulting in a report to the ministry — the children’s aid society responsible for the girl agreed with Hawk Homes, the group home operator, that she would be better cared for in a more costly individual program.

She was placed in a home where she was the only resident, and given round-the-clock caregivers. It wasn’t a panacea, and a government-appointed panel of experts that reviewed residential services in 2016 expressed concerns that such customized programs are not licensed by the ministry.

But the move eventually stabilized the girl’s behaviours.

Kiaras Gharabaghi, director of Ryerson University's school of child and youth care, says children's aid societies "know they don't have to remove kids (from residential care homes)" even if they are not well-placed"  because the private provider isn't going to kick the kid out.?

Kiaras Gharabaghi, director of Ryerson University’s school of child and youth care, says children’s aid societies “know they don’t have to remove kids (from residential care homes)” even if they are not well-placed” because the private provider isn’t going to kick the kid out. JIM RANKIN

Mark Williams, the group home’s executive director at the time, says approval for individualized programs usually only happens after criminal charges or when the operator threatens to discharge the youth after they have exhausted all other options.

“We saw very good results with such programs and saw cases where youth had serious occurrences in the triple digits in one year, reduce to single digits the next year,” adds Williams, who operated the individualized program.

But why didn’t the child protection system act sooner to spare the girl, her caregivers and other kids in the group home the trauma and injuries of her violent flare-ups?

Kiaras Gharabaghi, director of Ryerson University’s school of child and youth care, describes a system where children with complex needs end up losing.

Private group home operators rely on full beds to stay in business. He says they’re not likely to tell a children’s aid society they can’t handle a child with severe challenges for fear the society won’t refer other children to their care.

Societies “know they don’t have to remove kids — even if they are not well-placed — because the private provider isn’t going to kick the kid out,” says Gharabaghi, a member of the government-appointed panel that investigated residential care services in 2016.

“Then when things go really sour, they jump on the private provider and say, ‘Inadequate service,’ ” he adds.

But why didn’t the ministry detect the distress of the Quaker Rd. girl?

Serious occurrence reports must be sent to the ministry within 24 hours of an incident. The children’s aid society responsible for the child’s care is also notified — the Belleville-area Highland Shores society in the case of the Quaker Rd. girl.

Was anyone reading the pile of reports about her?

The inaction reflects a child protection system where the ministry, according to the 2016 report by the panel, is unable to keep track of kids in residential care, let alone ensure they’re being treated properly.

The ministry doesn’t even know how many kids are in Ontario’s 389 licensed group homes. A spokesperson says the ministry is working on a system to eventually figure that out.

In the meantime, the ministry keeps track of the number of beds. In the 2016-17 fiscal year, group homes had 2,914 beds, including 1,111 run by for-profit companies. Another 11,422 beds were in foster homes, including 2,005 managed as businesses.

An average of 14,000 kids were in Ontario’s child protection system last year, most taken from parents because of abuse or neglect. The ministry spends $1.5 billion on the system annually.

The panel’s 2016 report sharply criticized the ministry’s weak oversight role and its dismal efforts at data collection and analysis.

It described serious occurrence reports, for example, as key indicators of the quality of care children receive. (The reports are completed if there is a death, serious injury, runaway child, allegation of abuse or neglect, complaint about safety in the home or how it operates, or a disaster such as a power outage or flood.)

The reports get faxed to the ministry’s regional offices, where officials manually enter the data in a computer program before forwarding it to the central office in Toronto. With more than 23,000 reports annually, the inefficient process does nothing to increase the chances of the “under-utilized” reports being analyzed, the panel made clear.

The ministry responded in July 2017 with a reform blueprint for the residential care system, which includes group homes. Promises include setting minimum standards of care, developing a way to better monitor and keep track of where children are placed and moved, reducing the overrepresentation of Black and Indigenous children in care, and beefing up data collection, oversight and accountability.

After a fire at the Quaker Rd. girls home in February this year left a caregiver and a resident dead, the ministry began unannounced inspections of residential care sites. So far, 114 have been conducted. One led to the June closure of three privately run group homes in Thunder Bay, which were serving Indigenous children with mental health issues.

Inspections aside, the ministry doesn’t plan to fully implement the reforms until 2025. Gharabaghi has described the timeframe as unconscionably slow, insisting it leaves youths in unsafe care for the next eight years.

He also denounces ministry inaction on caregivers, who currently require no qualifications for the job. During the next eight years, the ministry’s goal is no more than “an action plan that will explore” the setting of minimum post-secondary education requirements and on-the-job training.

“There will continue to be opportunities for completely unqualified individuals to be entrusted with providing care and treatment to extremely vulnerable young people for many years to come,” Gharabaghi says.

“This has always been and continues to be absurd and perhaps abusive,” he adds, noting that numerous reports and inquiries over the past two decades have recommended minimum qualifications. “Why does this ministry lack the courage to move forward now?”

Psychologist Charlie Menendez worked with youth and staff at Hawk group homes near Lindsay, Ont. He says of the provincial system: "Staff need to become more effective as mental health professionals."

Psychologist Charlie Menendez worked with youth and staff at Hawk group homes near Lindsay, Ont. He says of the provincial system: “Staff need to become more effective as mental health professionals.” JIM RANKIN

The lack of qualifications keeps wages low — hovering at $14.50 an hour — in a sector where 70 per cent of the workers are women, according to Gharabaghi’s research. The median age is 25.

Young staff who have the training and a natural ability to be excellent front-line workers leave after six months or a year because of low wages and lack of support to do the job safely, says Charlie Menendez, a child psychologist who has worked with group home kids, including those on Quaker Rd.

“Staff need to become more effective as mental health professionals,” he says. “For this to happen they need to have the skills and capacities to do the job and they need to be supported appropriately by effective systems around them.

“Currently there is too much focus on protecting the kids from the staff, and not enough focus on helping the staff help the kids,” he says.

The expert panel also noted a growing use of unlicensed homes, like the one set up for the Highland Shores girl. It found that these customized programs often have untrained live-in staff and little quality-of-care oversight. It noted that in 2015, a child died during a physical restraint in one of those homes.

The panel urged the government to create special licences for these customized settings.

In the Highland Shores case, one incident caused the Kawartha children’s aid society to investigate.

It involved a struggle in which the girl bit a caregiver and the caregiver bit her back. The caregiver was fired.

The Kawartha society recommended that Williams, the home’s operator, increase daytime staffing levels from one to two caregivers. He replied that Highland Shores children’s aid “only provides funding for one staff at the home,” according to a Kawartha report to the ministry obtained by the Star though a freedom-of-information request.

Eventually, the girl’s behaviour stabilized enough to allow her to live independently, thanks largely to the leadership of Stephanie Stokes.


Stokes had walked into Hawk Homes in the fall of 2015 with a child and youth worker diploma from Loyalist College in Belleville. Stokes, 26, formed an instant connection with the troubled teen on her first day at Quaker Rd., when the girl tried to strangle her.

“It just clicked from that first day when she put me in that chokehold. I was like ‘What are you doing, kid? I’m not scared of you,’ ” says Stokes, who earned the nickname “happy hippie” because of her long hair and laid-back demeanour. “And she’s like, ‘OK, you’re cool. Whatever. Can’t scare ya.’ ”

Stokes worked mostly overnight and evening shifts, eventually developing a rapport with the girl that helped her get some sleep.

The girl was 17 when she was moved to a house on her own.

“Aggression went from all the time to pretty much nothing,” says Stokes, who led the girl’s caregiving team. “She needed consistency. She did not like having a bunch of kids around. She wanted the attention and to know that she was safe.”

The girl liked animals, so part of the program had her working on a farm with horses. Soon, she left a segregated school program to attend a regular high school in Lindsay.

In September 2016, when Hawk Homes closed because of financial stress and ministry scrutiny over quality-of-care concerns, the girl’s individual program also ended.

Stokes couldn’t bear to see the girl moved to another placement. So she offered to foster the girl in her own home. The girl lived with Stokes for almost a year.

She turned 18 last summer and is living on her own in the community. Stokes still speaks to her by phone almost every day.

“She had issues, but she was a great kid,” Stokes says. “She was the most kind-hearted, empathetic kid I have ever worked with. That’s why I stayed.

“She was scared. She wanted love. That’s all she ever wanted.”

Related stories:

CBC – ‘More than a tragedy’ Ontario child advocate says of youth who die when taken into care – 2017-11-28

‘More than a tragedy’ Ontario child advocate says of youth who die when taken into care

'More than a tragedy' Ontario child advocate says of youth who die when taken into care. Irwin Elman says he still wants inquests for every child who dies while in care of child welfare.

By Matt Prokopchuk, CBC News Posted: Nov 28, 2017 7:15 AM ET Last Updated: Nov 28, 2017 7:15 AM ET

Ontario's child and youth advocate Irwin Elman says a coroner's review of 11 young people who died while in care of child welfare services has to have the children at its core.

Ontario’s child and youth advocate Irwin Elman says a coroner’s review of 11 young people who died while in care of child welfare services has to have the children at its core.

Ontario’s child and youth advocate Irwin Elman says a coroner’s review of 11 young people who died while in care of child welfare services has to have the children at its core. (CBC)

The review to explore the deaths of 11 young people in Ontario who died while in the care of child welfare agencies must, at its core, focus on the children’s stories, according to the province’s advocate for children and youth.

Ontario’s chief coroner, Dirk Huyer, confirmed that the months-long “expert review” will probe how the youth were cared for when they were placed in homes or facilities away from their communities, and issues that may have arisen.

Grassy Narrows teen’s death to be part of ‘expert review’ of youth who died in child welfare care

“I think it’s really important,” Children and Youth Advocate Irwin Elman said of the review. “I’m not ready to say it’s something that should happen instead of an inquest [but] I will support it.” Huyer said the review does not preclude inquests from being called into any of the young people’s deaths, something Elman also acknowledged.

Elman said he wants to make sure “the children, through this process, are honoured.”

“That means their lives are explored with care, with honour and wonder about what we can do so that others don’t face a similar fate.”

Elman confirmed that he is not one of the seven members of the panel who will be charged with taking a closer look at how the young people — seven of whom are Indigenous and from the northwest part of the province — died, what systemic issues may have been present as well as coming up with recommendations. He said, however, that he will be involved in other ways.

“If families require advocacy support in this process, the coroner’s asked us to be available to them,” he said. “We would start by ensuring … that families have, if they’re going to be involved, have emotional support … because this is an arduous process.”

Elman added that his office will also help facilitate discussion and publicize any recommendations that come out of the review.

‘More than a tragedy’

While Elman said he still wants to see an inquest called “every time a child dies in a residential care facility,” he added that this process still has the potential to shed some light on an important issue.

“For me, when the state … says to a child, ‘you are coming with us for your own safety and protection,’ that’s a covenant we make with that child,” he said. “When those children, who have had a promise made to them by the province die … this is more than a tragedy.”

“It’s a broken covenant to a child.”

Azraya Kokopenace
Azraya Kokopenace, 14, was found dead in Kenora in April, 2016, two days after she walked away from the Lake of the Woods Hospital. Her family is calling for an inquest. (Marlin Kokopenace/Facebook)

One of the deaths being examined by the panel is that of Azraya Kokopenace, a 14-year-old from Grassy Narrows First Nation who was found dead in Kenora in 2016, two days after police dropped her off at the local hospital.

At the time, she was, by her own request, in the care of an agency in the city so she could receive counselling after the death of her brother Calvin in 2014 from mercury poisoning — a decades-old, and ongoing, heath crisis in Grassy Narrows.

Azraya’s family has been calling for an inquest into her death; their lawyer told CBC News that hasn’t changed in light of the coroner’s decision to hold the panel review, but the family will cooperate with it. Elman said he echoes those sentiments.

“This process (the panel review) is a collective process, bringing young people who had died together, in some ways,” Elman said. “For me, the process elevates their voice but when you do that, you can sometimes miss — and that would be my worry — the distinct voice of individual children.”

“If her voice — and her family feels her voice — is not heard in the review’s report, then I think another process is necessary to honour her,” he continued. “An inquest, from my point of view, can do that.”

The panel’s report is expected to be complete in the spring or summer of 2018.

Related stories:

With files from Jody Porter

Mothercraft – Child & Youth Mental Health Services under MCYS

Child & Youth Mental Health

Data dictionaries contain definitions for the data elements on which MCYS-funded and -operated agencies are required to report quarterly. The dictionaries are organized by funding code. See the Data Elements page for more information on Mothercraft’s role in ensuring that the data elements are collected and reported consistently across the children and youth services sector.

Type Document / Resource Size
Download PDF A505 – Residential Placement Advisory Committee DE Definitions 2013-14Residential Placement Advisory Committee, Service Data Element Definitions, 2013-14 (Apr 2013) 23.66 KB
Download PDF A508 – Children’s Comm Support – Other DE Definitions 2013-14Children’s Community Support – Other, Service Data Element Definitions, 2013-14 (Apr 2013) 39.20 KB
Download PDF A555 – Child and Family Intervention – Operating – Residential DE Definitions 2013-14Child and Family Intervention – Operating – Residential, Service Data Element Definitions, 2013-14 (Apr 2013) 28.29 KB
Download PDF A556 – Child and Family Intervention – Operating – Non-Residential DE Definitions 2013-14Child and Family Intervention – Operating – Non-Residential, Service Data Element Definitions, 2013-14 (Apr 2013) 25.85 KB
Download PDF A559 – Intensive Child and Family Services DE Definitions 2013-14Intensive Child and Family Services, Service Data Element Definitions, 2013-14 (Apr 2013) 30.33 KB
Download PDF A560 – Mobile Crisis DE Definitions 2013-14Mobile Crisis, Service Data Element Definitions, 2013-14 (Apr 2013) 30.12 KB
Download PDF A562 – CMH 0-6 DE Definitions 2013-14Children’s Mental Health (CMH) 0-6, Service Data Element Definitions, 2013-14 (Apr 2013) 30.31 KB
Download PDF A566 – Section 23 Classrooms DE Defintions 2013-14Section 23 Classrooms, Service Data Element Definitions, 2013-14 (Apr 2013) 28.18 KB
Download PDF A577 – Child Treatment – Operating – Secure DE Definitions 2013-14Child Treatment – Operating – Secure, Service Data Element Definitions, 2013-14 (Apr 2013) 28.10 KB
Download PDF A578 – Child Treatment – Residential DE Definitions 2013-14Child Treatment – Residential, Service Data Element Definitions, 2013-14 (Apr 2013) 28.07 KB
Download PDF A579 – Child Treatment – Operating – Non-Residential DE Definitions 2013-14Child Treatment – Operating – Non-Residential, Service Data Element Definitions, 2013-14 (Apr 2013) 25.63 KB
Download PDF A583 – CMH Outpatient Programs DE Definitions 2013-14Children’s Mental Health (CMH) Outpatient Programs, Service Data Element Definitions, 2013-14 (Apr 2013) 29.21 KB

Cory Doctorow: How stupid laws and benevolent dictators can ruin the decentralized web, too – 20160608

Keynote address to the Internet Archive Decentralized Web Summit

So, as you might imagine, I’m here to talk to you about dieting advice. If you ever want to go on a diet, the first thing you should really do is throw away all your Oreos.

It’s not that you don’t want to lose weight when you raid your Oreo stash in the middle of the night. It’s just that the net present value of tomorrow’s weight loss is hyperbolically discounted in favor of the carbohydrate rush of tonight’s Oreos. If you’re serious about not eating a bag of Oreos your best bet is to not have a bag of Oreos to eat. Not because you’re weak willed. Because you’re a grown up. And once you become a grown up, you start to understand that there will be tired and desperate moments in your future and the most strong-willed thing you can do is use the willpower that you have now when you’re strong, at your best moment, to be the best that you can be later when you’re at your weakest moment.

And this has a name: It’s called a Ulysses pact. Ulysses was going into Siren-infested waters. When you go into Siren-infested waters, you put wax in your ears so that you can’t hear what the Sirens are singing, because otherwise you’ll jump into the sea and drown. But Ulysses wanted to hear the Sirens. And so he came up with a compromise: He had his sailors tie him to the mast, so that when he heard the call of the Sirens, even though he would beg and gibber and ask them to untie him, so that he could jump into the sea, he would be bound to the mast and he would be able to sail through the infested waters.

This is a thing that economists talk about all the time, it’s a really critical part of how you build things that work well and fail well. Now, building a Web that is decentralized is a hard thing to do, and the reason that the web ceases to be decentralized periodically is because it’s very tempting to centralize things. There are lots of short term gains to be had from centralizing things and you want to be the best version of yourself, you want to protect your present best from your future worst.

The reason that the Web is closed today is that people just like you, the kind of people who went to Doug Engelbart’s demo in 1968, the kind of people who went to the first Hackers conference, people just like you, made compromises, that seemed like the right compromise to make at the time. And then they made another compromise. Little compromises, one after another.

And as humans, our sensory apparatus is really only capable of distinguishing relative differences, not absolute ones. And so when you make a little compromise, the next compromise that you make, you don’t compare it to the way you were when you were fresh and idealistic. You compare it to your current, “stained” state. And a little bit more stained hardly makes any difference. One compromise after another, and before you know it, you’re suing to make APIs copyrightable or you’re signing your name to a patent on one-click purchasing or you’re filing the headers off of a GPL library and hope no one looks too hard at your binaries. Or you’re putting a backdoor in your code for the NSA.

And the thing is: I am not better than the people who made those compromises. And you are not better than the people who made those compromises. The people who made those compromises discounted the future costs of the present benefits of some course of action, because it’s easy to understand present benefits and it’s hard to remember future costs.

You’re not weak if you eat a bag of Oreos in the middle of the night. You’re not weak if you save all of your friends’ mortgages by making a compromise when your business runs out of runway. You’re just human, and you’re experiencing that hyperbolic discounting of future costs because of that immediate reward in the here and now. If you want to make sure that you don’t eat a bag of Oreos in the middle of the night, make it more expensive to eat Oreos. Make it so that you have to get dressed and find your keys and figure out where the all-night grocery store is and drive there and buy a bag of Oreos. And that’s how you help yourself in the future, in that moment where you know what’s coming down the road.

The answer to not getting pressure from your bosses, your stakeholders, your investors or your members, to do the wrong thing later, when times are hard, is to take options off the table right now. This is a time-honored tradition in all kinds of economic realms. Union negotiators, before they go into a tough negotiation, will say: “I will resign as your negotiator, before I give up your pension.” And then they sit down across the table from the other side, and the other side says “It’s pensions or nothing”. And the union leaders say: “I hear what you’re saying. I am not empowered to trade away the pensions. I have to quit. They have to go elect a new negotiator, because I was elected contingent on not bargaining away the pensions. The pensions are off the table.”

Brewster has talked about this in the context of code, he suggested that we could build distributed technologies using the kinds of JavaScript libraries that are found in things like Google Docs and Google Mail, because no matter how much pressure is put on browser vendors, or on technology companies in general, the likelihood that they will disable Google Docs or Google Mail is very, very low. And so we can take Google Docs hostage and use it as an inhuman shield for our own projects.

The GPL does this. Once you write code, with the GPL it’s locked open, it’s irrevocably licensed for openness and no one can shut it down in the future by adding restrictive terms to the license. The reason the GPL works so well, the reason it became such a force for locking things open, is that it became indispensable. Companies that wanted to charge admission for commodity components like operating systems or file editors or compilers found themselves confronted with the reality that there’s a huge difference between even a small price and no price at all, or no monetary price. Eventually it just became absurd to think that you would instantiate a hundred million virtual machines for an eleventh of a second and get a license and a royalty for each one of them.

And at that point, GPL code became the only code that people used in cloud applications in any great volume, unless they actually were the company that published the operating system that wasn’t GPL’d. Communities coalesced around the idea of making free and open alternatives to these components: GNU/Linux, Open- and LibreOffice, git, and those projects benefited from a whole bunch of different motives, not always the purest ones. Sometimes it was programmers who really believed ethically in the project and funded their own work, sometimes talent was tight and companies wanted to attract programmers, and the way that they got them to come through the door is by saying: “We’ll give you some of your time to work on an ethical project and contribute code to it.”

Sometimes companies got tactical benefits by zeroing out the margins on their biggest competitor’s major revenue stream. So if you want to fight with Microsoft, just make Office free. And sometimes companies wanted to use but not sell commodity components. Maybe you want to run a cloud service but you don’t want to be in the operating system business, so you put a bunch of programmers on making Linux better for your business, without ever caring about getting money from the operating system. Instead you get it from the people who hire you to run their cloud.

Everyone of those entities, regardless of how they got into this situation of contributing to open projects, eventually faced hard times, because hard times are a fact of life. And systems that work well, but fail badly, are doomed to die in flames. The GPL is designed to fail well. It makes it impossible to hyperbolically discount the future costs of doing the wrong thing to gain an immediate benefit. When your investor or your acquisition suitor or your boss say “Screw your ethics, hippie, we need to make payroll”, you can just pull out the GPL and say: “Do you have any idea how badly we will be destroyed if we violate copyright law by violating the GPL?”

It’s why Microsoft was right to be freaked out about the GPL during the Free and Open Source wars. Microsoft’s coders were nerds like us, they fell in love with computers first, and became Microsoft employees second. They had benefited from freedom and openness, they had cated out BASIC programs, they had viewed sources, and they had an instinct towards openness. Combining that with the expedience of being able to use FLOSS, like not having to call a lawyer before you could be an engineer, and with the rational calculus, that if they made FLOSS, that when they eventually left Microsoft they could keep using the code that they had made there, meant that Microsoft coders and Microsoft were working for different goals. And the way they expressed that was in how they used and licensed their code.

This works so well that for a long time, nobody even knew if the GPL was enforceable, because nobody wanted to take the risk of suing and setting a bad precedent. It took years and years for us to find out in which jurisdictions we could enforce the GPL.

That brings me to another kind of computer regulation, something that has been bubbling along under the surface for a long time, at least since the Open Source wars, and that’s the use of Digital Rights Management (DRM) or Digital Restrictions Management, as some people call it. This is the technology that tries to control how you use your computer. The idea is that you have software on the computer that the user can’t override. If there is remote policy set on that computer that the user objects to, the computer rejects the user’s instruction in favor of the remote policy. It doesn’t work very well. It’s very hard to stop people who are sitting in front of a computer from figuring out how it works and changing how it works. We don’t keep safes in bank robbers’ living rooms, not even really good ones.

But we have a law that protects it, the Digital Millennium Copyright Act (DMCA), it’s been around since 1998 and it has lots of global equivalents like section 6 of the EUCD in Europe, implemented all across the EU member states. In New Zealand they tried to pass a version of the DMCA and there were uprisings and protests in the streets, they actually had to take the law off the books because it was so unpopular. And then the Christchurch earthquake hit and a member of parliament reintroduced it as a rider to the emergency relief bill to dig people out of the rubble. In Canada it’s Bill C-11 from 2011. And what it does is, it makes it a felony to tamper with those locks, a felony punishable by 500,000 dollars fine and five years in jail for a first offense. It makes it a felony to do security auditing of those locks and publish information about the flaws that are present in them or their systems.

This started off as a way to make sure that people who bought DVDs in India didn’t ship them to America. But it is a bad idea whose time has come. It has metastasized into every corner of our world. Because if you put just enough DRM around a product that you can invoke the law, then you can use other code, sitting behind the DRM, to control how the user uses that product, to extract more money. GM uses it to make sure that you can’t get diagnostics out of the car without getting a tool that they license to you, and that license comes with a term that says you have to buy parts from GM, and so all repair shops for GM that can access your diagnostic information have to buy their parts from GM and pay monopoly rents.

We see it in insulin pumps, we see it in thermostats and we see it in the “Internet of Things rectal thermometer”, which debuted at CES this year, which means we now have DRM restricted works in our asses. And it’s come to the web. It’s been lurking in the corners of the web for a long time. But now it’s being standardized at the World Wide Web Consortium (W3C) to something called Encrypted Media Extensions (EME). The idea of EME is that there is conduct that users want to engage in that no legislature in the world has banned, like PVR’ing their Netflix videos. But there are companies that would prefer that conduct not to be allowed. By wrapping the video with just enough DRM to invoke the DMCA, you can convert your commercial preference to not have PVRs (which are no more and no less legal than the VCR was when in 1984 the Supreme Court said you can record video off your TV) into something with the force of law, whose enforcement you can outsource to national governments.

What that means, is that if you want to do interoperability without permission, if you want to do adversarial interoperability, if you want to add a feature that the manufacturer or the value chain doesn’t want, if you want to encapsulate Gopher inside of the Web to launch a web browser with content form the first day, if you want to add an abstraction layer that lets you interoperate between two different video products so that you can shop between them and find out which one has the better deal, that conduct, which has never been banned by a legislature, becomes radioactively illegal.

It also means, that if you want to implement something that users can modify, you will find yourself at the sharp end of the law, because user modifiability for the core components of the system is antithetical to its goals of controlling user conduct. If there’s a bit you can toggle that says “Turn DRM off now”, then if you turn that bit off, the entire system ceases to work. But the worst part of all is that it makes browsers into no-go zones for security disclosures about vulnerabilities in the browser, because if you know about a vulnerability you could use it to weaken EME. But you could also use it to attack the user in other ways.

Adding DRM to browsers, standardizing DRM as an open standards organization, that’s a compromise. It’s a little compromise, because after all there’s already DRM in the world, and it’s a compromise that’s rational if you believe that DRM is inevitable. If you think that the choice is between DRM that’s fragmented or DRM that we get a say in, that we get to nudge into a better position, then it’s the right decision to make. You get to stick around and do something to make it less screwed up later, as opposed to being self-marginalized by refusing to participate at all.

But if DRM is inevitable, and I refuse to believe that it is, it’s because individually, all across the world, people who started out with the best of intentions made a million tiny compromises that took us to the point where DRM became inevitable, where the computers that are woven into our lives, with increasing intimacy and urgency, are designed to control us instead of being controlled by us. And the reasons those compromises were made is because each one of us thought that we were alone and that no one would have our back, that if we refuse to make the compromise, the next person down the road would, and that eventually, this would end up being implemented, so why not be the one who makes the compromise now.

They were good people, those who made those compromises. They were people who were no worse than you and probably better than me. They were acting unselfishly. They were trying to preserve the jobs and livelihoods and projects of people that they cared about. People who believed that others would not back their play, that doing the right thing would be self-limiting. When we’re alone, and when we believe we’re alone, we’re weak.

It’s not unusual to abuse standards bodies to attain some commercial goal. The normal practice is to get standards bodies to incorporate your patents into a standard, to ensure that if someone implements your standard, you get a nickel every time it ships. And that’s a great way to make rent off of something that becomes very popular. But the W3C was not armtwisted about adding patents back into standards. That’s because the W3C has the very best patents policy of any standards body in the world. When you come to the W3C to make a standard for the web, you promise not to use your patents against people who implement that standard. And the W3C was able to make that policy at a moment in which it was ascendant, in which people were clamoring to join it, in which it was the first moments of the Web and in which they were fresh.

The night they went on a diet, they were able to throw away all the Oreos in the house. They were where you are now, starting a project that people around the world were getting excited about, that was showing up on the front page of the New York Times. Now that policy has become the ironclad signifier of the W3C. What’s the W3C? It’s the open standards body that’s so open, that you don’t get to assert patents if you join it. And it remains intact.

How will we keep the DMCA from colonizing the Locked Open Web? How will we keep DRM from affecting all of us? By promising to have each others’ backs. By promising that by participating in the Open Web, we take the DMCA off the table. We take silencing security researchers, we take blocking new entrances to the market off the table now, when we are fresh, when we are insurgent, before we have turned from the pirates that we started out as into the admirals that some of us will become. We take that option off the table.

The EFF has proposed a version of this at the W3C and at other bodies, where we say: To be a member, you have to promise not to use the DMCA to aggress against those, who report security vulnerabilities in W3C standards, and people who make interoperable implementations of W3C standards. We’ve also proposed that to the FDA, as a condition of getting approval for medical implants, we’ve asked them to make companies promise in a binding way never to use the DMCA to aggress against security researchers. We’ve taken it to the FCC, and we’re taking it elsewhere. If you want to sign an open letter to the W3C endorsing this, email me: cory@eff.org

But we can go further than that, because Ulysses pacts are fantastically useful tools for locking stuff open. It’s not just the paper that you sign when you start your job, that takes a little bit of money out of your bank account every month for your 401k, although that works, too. The U.S. constitution is a Ulysses pact. It understands that lawmakers will be corrupted and it establishes a principal basis for repealing the laws that are inconsistent with the founding principles as well as a process for revising those principles as need be.

A society of laws is a lot harder to make work than a society of code or a society of people. If all you need to do is find someone who’s smart and kind and ask them to make all your decisions for you, you will spend a lot less time in meetings and a lot more time writing code. You won’t have to wrangle and flame or talk to lawyers. But it fails badly. We are all of us a mix of short-sighted and long-term, depending on the moment, our optimism, our urgency, our blood-sugar levels…

We must give each other moral support. Literal moral support, to uphold the morals of the Decentralized Web, by agreeing now what an open internet is and locking it open. When we do that, if we create binding agreements to take certain kinds of conduct off the table for anything that interoperates with or is part of what we’re building today, then our wise leaders tomorrow will never be pressurized to make those compromises, because if the compromise can’t be made, there is no point in leaning on them to make it.

We must set agreements and principles that allow us to resist the song of the Sirens in the future moments of desperation. And I want to propose two key principles, as foundational as life, liberty, and the pursuit of happiness or the First Amendment:

1) When a computer receives conflicting instructions from its owner and from a remote party, the owner always wins.

Systems should always be designed so that their owners can override remote instructions and should never be designed so that remote instructions can be executed if the owner objects to them. Once you create the capacity for remote parties to override the owners of computers, you set the stage for terrible things to come. Any time there is a power imbalance, expect the landlord, the teacher, the parent of the queer kid to enforce that power imbalance to allow them to remotely control the device that the person they have power over uses.

You will create security risks, because as soon as you have a mechanism that hides from the user, to run code on the user’s computers, anyone who hijacks that mechanism, either by presenting a secret warrant or by breaking into a vulnerability in the system, will be running in a privileged mode that is designed not to be interdicted by the user.

If you want to make sure that people show up at the door of the Distributed Web asking for backdoors, to the end of time, just build in an update mechanism that the user can’t stop. If you want to stop those backdoor requests from coming in, build in binary transparency, so that any time an update ships to one user that’s materially different from the other ones, everybody gets notified and your business never sells another product. Your board of directors will never pressurize you to go along with the NSA or the Chinese secret police to add a backdoor, if doing so will immediately shut down your business.

Throw away the Oreos now.

Let’s also talk about the Computer Fraud and Abuse Act. This is the act that says if you exceed your authorization on someone else’s computer, where that authorization can be defined as simply the terms of service that you click through on your way into using a common service, you commit a felony and can go to jail. Let’s throw that away, because it’s being used routinely to shut down people who discover security vulnerabilities in systems.

2) Disclosing true facts about the security of systems that we rely upon should never, ever be illegal.

We can have normative ways and persuasive ways of stopping people from disclosing recklessly, we can pay them bug bounties, we can have codes of conduct. But we must never, ever give corporations or the state the legal power to silence people who know true things about the systems we entrust our lives, safety, and privacy to.

These are the foundational principles. Computers obey their owners, true facts about risks to users are always legal to talk about. And I charge you to be hardliners on these principles, to be called fanatics. If they are not calling you puritans for these principles you are not pushing hard enough. If you computerize the world, and you don’t safeguard the users of computers form coercive control, history will not remember you as the heroes of progress, but as the blind handmaidens of future tyranny.

This internet, this distributed internet that we are building, the Redecentralization of the Internet, if it ever succeeds, will someday fail, because everything fails, because overwhelmingly, things are impermanent. What it gives rise to next, is a function of what we make today. There’s a parable about this:

The state of Roman metallurgy in the era of chariots, determined the wheel base of a Roman chariot, which determined the width of the Roman road, which determined the width of the contemporary road, because they were built atop the ruins of the Roman roads, which determined the wheel base of cars, which determined the widest size that you could have for a container that can move from a ship, to a truck, to a train, which determined the size of a train car, which determined the maximum size of the Space Shuttle’s disposable rockets.

Roman metallurgy prefigured the size of the Space Shuttle’s rockets.

This is not entirely true, there are historians who will explain the glosses in which it’s not true. But it is a parable about what happens when empires fall. Empires always fall. If you build a glorious empire, a good empire, an empire we can all be proud to live in, it will someday fall. You cannot lock it open forever. The best you can hope for is to wedge it open until it falls, and to leave behind the materials, the infrastructure that the people who reboot the civilization that comes after ours will use to make a better world.

A legacy of technology, norms and skills that embrace fairness, freedom, openness and transparency, is a commitment to care about your shared destiny with every person alive today and all the people who will live in the future.

[Transcript by Jonke Suhr]

How to protect the future web from its founders’ own frailty

Can data shape the future of mental health support? – The Guardian 20160907

From: Can data shape the future of mental health support? – The Guardian 20160907

Open data is being used to design resources for people with mental health conditions to help them find the right support

Head on digital screen.

If you’re experiencing a mental health issue, one of the people you probably least want to speak to about it is your employer. Disclosing depression or anxiety has long been seen as the last workplace taboo, for fear of repercussions. This is despite the existence of the Equality Act 2010, which protects employees with physical and mental disabilities from discrimination.

But just over a third of workers with a mental health condition discuss it with their employer, according to a survey of 1,388 employees carried out by Willis PMI Group, one of the UK’s largest providers of employee healthcare and risk management services. The research found that 30% of respondents were concerned that they wouldn’t receive adequate support, 28% believed their employer wouldn’t understand, and 23% feared that disclosing it would lead to management thinking less of them.

A culture of fear and silence can have a huge impact on productivity – the charity Mind estimates [pdf] that mental ill health costs the economy £70bn a year. The challenge is that seeking help involves taking ownership of the problem, says Mark Brown, development director of social enterprise Social Spider and founder of the now defunct mental health and wellbeing magazine One in Four. And finding support online can be a time-consuming and frustrating experience.

“Just serving up ever great slabs of information – the internet is awash with it – isn’t going to help anyone to know what to do,” says Brown. “We often confuse the provision of information with the solving of problems. Knowing information is different from knowing how to put that information into action.”

Brown believes that bringing together information with public and open data into a single digital space is one way that could innovate how advice is delivered.

Plexus is aiming to achieve just this. Built by the digital studio M/A, with funding from the Open Data Institute, the knowledge base is being used to design resources for people with mental health conditions, their families, and even employers, to find support available in local areas, seek advice on how best to cope with returning to work after a period off and understand employee rights and employer responsibilities.

Plexus has pooled data from a couple of dozen organisations including NHS Choices, Department for Work and Pensions, the Office for National Statistics and Citizens Advice. In some cases the information has been pulled from APIs; in other instances it has been scraped using web data platform import.io.

The first tool Plexus developed is a chatbot called Grace, which is currently in beta testing. It enables users to record thoughts and feelings anonymously, receive feedback in the form of a newsletter and log in to an online dashboard to see a more detailed analysis, including whether there are any patterns in mood emerging over a period of time. The tool also offers guidance from the various governmental and charity websites under easy-to-navigate sections, such as legal rights and preparing for work.

“Through machine learning, Grace will intuitively know when our users are mostly likely to want to speak with us, be able to see the positive and negative nature of the user’s reply, and adapt the questions to encourage more positive responses,” explains Martin Vowles, creative director and co-founder of M/A. “We’re hoping this approach will allow us to offer a unique tool to each user which helps them understand and develop their mental wellbeing.”

Brown says that the potential for machine learning to tailor information and services is exciting. “It’s very good at looking at big piles of data for patterns. When we know certain things to be correct from one dataset, it can begin to make guesses about lots of other things based on what the machine is being fed.”

The sensitive nature of data being submitted by users on a platform like Grace, though, means many people are likely to be uneasy about their data being made accessible. To get round this, Plexus allows users to decide how their data is shared, with data licences lasting between 13 and 26 weeks. Vowles hopes that “as users become more trusting of Grace and what it can do for them, they’ll become more trusting with [us] using their anonymised personal data.”

Plexus aims to release a series of open datasets, including qualitative, quantitative and information on resources accessed by Grace users, to enable NGOs and local authorities to understand the country’s mental health provision. It’s hoped that they’d then use the knowledge to devise new strategies and ensure targets are met and resources and services available in local areas are of an acceptable standard.

There are also plans to make certain data available to employers, but “this has to be on the employee’s terms”. Vowles imagines that involving employers in the process of receiving support could allow them to get a clearer picture of mental health in the workplace. They could then adapt to make employees feel more comfortable and ensure their business has adequate support in place.

The potential to use open data to shape how future mental health support is delivered is an area that has been underexplored. At the end of last year, the Royal Society of Arts launched an interactive platform with Mind that allows members of the public to find out how well local health providers are looking after people with mental health conditions. The full dataset is available to download and includes data extracted from Public Health England, as well as metrics such as percentage of people with a mental health condition in employment in local areas. Plexus, however, is the first tool to use open data with the aim of providing people with a holistic view of their mental wellbeing.

Brown supports the idea of using open datasets and combine them, but stresses that any tool or platform has to benefit users. The data and information must be digestible and it needs to help them understand and take away from it what they need.

“It’s often extremely easy to forget that people with mental health difficulties are people first and foremost – not objects or problems.”

The United Nations Secretary-General’s High-Level Panel on Access to Medicines Releases Final Report – 20160914

The United Nations Secretary-General’s High-Level Panel on Access to Medicines Releases Final Report

High-Level Panel on Access to Medicines

Letter from Panel Co-Chairs: Click Here

THE FINAL REPORT (PDF)

UNITED NATIONS SECRETARY-GENERAL’S HIGH-LEVEL PANEL ON ACCESS TO MEDICINES CALLS FOR NEW DEAL TO CLOSE THE HEALTH INNOVATION AND ACCESS GAP

Whether it’s the rising price of the EpiPen, or new outbreaks of diseases, like Ebola, Zika and yellow fever, the rising costs of health technologies and the lack of new tools to tackle health problems, like antimicrobial resistance, is a problem in rich and poor countries alike.

According to a High-Level Panel convened to advise the UN Secretary-General on improving access to medicines, the world must take bold new approaches to both health technology innovation and ensuring access so that all people can benefit from the medical advances that have dramatically improved the lives of millions around the world in the last century.

For decades, many international treaties and national constitutions have enshrined the fundamental right to health and the right to share in the benefits of scientific advancements.  Yet, while the world is witnessing the immense potential of science and technology to advance health care, gaps and failures in addressing disease burdens and emerging diseases in many countries and communities remain. The misalignment between the right to health on the one hand and intellectual property and trade on the other, fuel this tension.

The UN Secretary-General established the High-Level Panel to propose solutions for addressing the incoherencies between international human rights, trade, intellectual property rights and public health objectives. The report recommendations come at the end of a ten-month process for the Panel under the leadership of Ruth Dreifuss and the former President of the Swiss Confederation and Festus Mogae, the former President of the Republic of Botswana.

“Policy incoherencies arise when legitimate economic, social and political interests and priorities are misaligned or in conflict with the right to health,” said President Ruth Dreifuss. “On the one hand, governments seek the economic benefits of increased trade.  On the other, the imperative to respect patents on health technologies could, in certain instances, create obstacles to the public health objectives and the right to health.”

The Panel has formulated a set of concrete recommendations to help improve research and development of health technologies and people’s access to vital therapies that are currently priced out-of-reach of patients and governments alike. The Panel’s report points out that the cost of health technologies are putting a strain on both rich and poor countries.

“With no market incentives, there is an innovation gap in diseases that predominantly affect neglected populations,  rare diseases and a crisis particularly with antimicrobial resistance, which poses a threat to humanity,” said Malebona Precious Matsoso, Director General of the National Department of Health of South Africa. “Our report calls on governments to negotiate global agreements on the coordination, financing and development of health technologies to complement existing innovation models, including a binding R&D Convention that delinks the costs of R&D from end prices.”

The Panel suggested that initially governments should form a working group to begin negotiating a Code of Principles for Biomedical R&D, and report annually on their progress in negotiating and implementing the Code in preparation for negotiating the Convention.

The Panel examined the way in which the application of the flexibilities found in the WTO Agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS) has facilitated access to health technologies, and how WTO Members can tailor national intellectual property law, competition law, government procurement and drug regulatory laws and regulations to fulfil public health obligations.

The new report noted with grave concern reports of governments being subjected to undue political and economic pressure to forgo the use of TRIPS flexibilities. The Panel felt strongly that this pressure undermines the efforts of governments to meet their human rights and public health obligations and violates the integrity and legitimacy of the Doha Declaration.

“WTO Members must make full use of TRIPS flexibilities as reaffirmed by the Doha Declaration on TRIPS and Public Health.  This is essential to promote access to health technologies,” said Michael Kirby, member of the High-Level Panel and chair of the Expert Advisory Group. “In particular, governments and the private sector must refrain from explicit or implicit threats, tactics or strategies that undermine the right of WTO Members to use TRIPS flexibilities.  WTO Members must register complaints against undue political and economic pressure.  They need to take strong, effective measures against offending Members.”

Transparency was a recurring theme throughout the report of the High-Level Panel. The Panel repeatedly raised concerns regarding the negative impact of insufficient transparency on both health technology innovation and access. The Panel was also critical of the lack of transparency surrounding bilateral free trade and investment negotiations. The Panel views transparency as a core component of robust and effective accountability frameworks needed to hold all stakeholders responsible for the impact of their actions on innovation and access.

“A paradigm shift in transparency is needed to ensure that the costs of R&D, production, marketing, and distribution, as well as the end prices of health technologies are clear to consumers and governments,” said President Festus Mogae. “Governments should require manufacturers and distributors of health technologies to disclose these costs and the details of any public funding received in the development of health technologies, including tax credits, subsidies, and grants.”

The Panel also recommended the UN General Assembly convene a Special Session no later than 2018 on health technology innovation and access to agree on strategies and an accountability framework that will accelerate efforts towards promoting innovation and ensuring access in line with the 2030 Agenda for Sustainable Development.

Tribunal orders Canada, again, to comply with its ruling on First Nation child welfare – APTN National News 20160915

Tribunal orders Canada, again, to comply with its ruling on First Nation child welfare

The federal government continues to drag its heels on fully complying with the landmark decision of Canadian Human Rights Tribunal in January that found Ottawa discriminates against First Nation children on-reserve.

The tribunal ruled for far too long the feds funded First Nation children living on-reserve less than non-Indigenous children off of reserve.

It gave the government a list of areas that needed to fixed and issued a compliance order.

They did so again in April.

On Thursday, the tribunal did so again.

“It rests on INAC and the federal government to implement the panel’s findings and orders, and to clearly communicate how it is doing so,” the tribunal said Thursday.

The tribunal said it’s unclear who the feds have consulted with in the Indigenous community address the gaps.

“INAC has previously acknowledged that it does not have expertise in the provision of child and family services to First Nations. Therefore, the need to consult with experts in the field, including the Caring Society, should be a priority,” the tribunal said, referring to Cindy Blackstock, executive director of First Nations Child and Family Caring Society of Canada.

Blackstock, along with the Assembly of First Nations, were the ones who first brought the complaint against Ottawa about 10 years ago.

Blackstock said she was happy with the latest compliance order and was scheduled to hold a media conference in Ottawa Thursday afternoon.

The AFN also was happy the tribunal is keeping on the feds.

“But it is disappointing to see that Canada has to be pushed to respect human rights and end discrimination against First Nations children, said National Chief Perry Bellegarde. “Canada must be more transparent and work with us a on a better system to reform the federal First Nation child welfare program that is supported by fair funding based on real needs.”

The federal government is expected to release a statement later Thursday.

Animating geodemographic data with D3

What’s here

  • Some thoughts on the cost of establishing provincial bodies to design, deliver, and monitor public services whose geographic boundaries are misaligned with the national census
  • An extensive collection of geodemographic files
  • Some cool (at least I think so!) data visualizations involving sliders and animation

An aside

Since last time, I have been familiarizing myself with some of the more technical details of the Canadian government’s census and the administrative bodies that the Ontario government has established to plan, deliver, and monitor health and human services. In many instances, the geographic boundaries of these administrative bodies are difficult – if not impossible – to align with Canada’s census divisions, census subdivisions, dissemination areas, dissemination blocks, and other geographic units. This misalignment seriously challenges the use of expensive census data to address the social determinants of health and well-being in our public services. 1 The cost of this misalignment needs to figure more prominently and explicitly in any future decisions to establish new or modify existing administrative bodies. In this regard, the decision of the Ministry of Children and Youth to align the geographic boundaries of its five Integrated Service Regions (ISRs) and its thirty-three children and youth mental health Service Areas (CYMHSAs) with Ontario’s census divisions is commendable.

New geodemographic datasets

I have to admit that earlier posts have run the risk of presenting a somewhat disjointed view of the population projections and the geographic boundaries of administrative bodies that the Ontario government has established to plan, deliver, and monitor health and human services. So I want to consolidate some of my thinking and share a number of geodemographic datasets that will form the basis of future data analyses and visualizations. I have structured these datasets using three main categories – geographic unit, classification of age, and demographic variable – each with a number of sub-types:

  • Geographic Unit (4)
    • Statistics Canada
      • Census Division (CD)
    • Ontario Ministry of Children and Youth Services
      • Child and Youth Mental Health Service Area (CYMHSA)
      • Integrated Service Region (ISR)
    • Ontario Ministry of Health and Long Term Care
      • Local Health Integration Network (LHIN)
  • Classification of Age (5)
    • Life Cycle Grouping (LCG)
      • Child (0 – 14)
      • Youth (15 – 24)
    • Young Person
      • LCG-Child and LCG-combined (0 – 24)
      • CFSA-Child (0 – 17)
    • Transitional-Age Person
      • Emerging Adult (16 – 20)
  • Demographic Variable (7)
    • Population
    • Population Density
    • Population Share
    • Population Growth
      • Annual Growth
        • Change in number of persons
        • Rate of growth (%)
      • 5 Year Growth
        • Change in number of persons
        • Rate of growth (%)

We have met with most of these categories and sub-types before, or they carry their usual meanings here. One exception is the CFSA-Child (0 – 17) classification of age, which refers to the definition of “child” under the Child and Family Services Act in Ontario. CFSA-Child is central to any data analyses and visualizations relating to the CYMHSAs and ISRs defined by the province’s Ministry of Children and Youth.

We have compiled twenty-eight datasets in *.csv format that profile the different classifications of age for the years 2016 to 2041:2

Table 1. Datafiles for five classifications of age (above), 2016 to 2041 within a Geographic Unit x Demographic Variable tuple.
Geographic Unit
Demographic Variable CD CYMHSA ISR LHIN
Population data data data data
Population density data data data data
Population share data data data data
Growth
   Annual growth
       Change in number of persons data data data data
       Rate of growth (%) data data data data
   5-year growth
       Change in number of persons data data data data
       Rate of growth (%) data data data data

Sliders and animations

Of course, compiling all of these geodemographic datasets is only worthwhile if we can use them to make better decisions. In earlier posts, we have illustrated how to generate static maps of the geographic boundaries of administrative bodies like the MCYS’s CYMHSAs and ISRs, the MOHLTC’s LHINs, and we have introduced some limited capabilities for the user to interact (zoom, pan) with more dynamic maps.

Before leaving off here, I wanted to illustrate the use of sliders and animation to enhance the user’s abilities – not only to interact with maps – but to discern spatial-temporal patterns in the demographic data.

First, let’s introduce a slider that allows the user to move forward and backwards in time as we use a choropleth map to illustrate the rate of growth in the Child (0 – 14) population in Ontario’s fourty-nine Census Divisions, 2016 to 2041 [interactive version]:

Screen CDS GrowthPC Child slider
Figure 1. Choropleth with slider, illustrating annual rate of growth in Child (0 – 14) population by Census Division in Ontario, 2016 to 2041.

Second, let’s use animation to display the same data [interactive version]:

Screen CDS GrowthPC Child animation
Figure 2. Choropleth with animation, illustrating annual rate of growth in Child (0 – 14) population by Census Division in Ontario, 2016 to 2041.

Next time: I will provide a (fairly) friendly interface that will allow users to select among the 140 possible combinations of Classification of Age x Demographic Variable x Geographic Unit.

 

  1. For example, consider the challenge that Ontario’s Ministry of Finance confronts when it provides population projections for Ontario’s Local Health Integration Networks (LHINs). Except for a few cases, the boundaries of the LHINs do not conform to those of Census Divisions (CDs) or Census Subdivisions (CSDs). Thus, to take advantage of Statistics Canada’s annual updates of population projections to revise the basic demographic profiles of the LHINs, the Ministry of Finance must resort to a number of fiddles, depending on how a particular LHIN splits one or more CDs and/or CSDs. Case 1: If the LHIN consists of intact CDs (the Erie St. Clair LHIN), the Ministry of Finance’s CD-level projections for each CD are aggregated. Case 2: If the LHIN does not include any part of Toronto, York, and Peel AND its boundary splits CDs but not CSDs (the Champlain, South East, North-East, and North-West LHINs), the share-of-growth method is used. Case 3:  If the LHIN boundary splits CSDs (as well as CDs) in Toronto, York, and Peel (the Central West, Mississauga Halton, Toronto Central, and Central East LHINs), the share-of-growth method is used based on the growth of Dissemination Areas. Case 4: If the LHIN boundary splits CSDs (as well as CDs) in CDs other than Toronto, York, and Peel (the South West, Waterloo-Wellington, Hamiltion Niagara Haldimand Brant, Central, and North Simcoe Muskoka LHINs), the constant-share method is used. Additional iterative prorating procedure is required to deal with the age-sex structure of CSDs and split CSDs. All of this merely to update basic population projections! Imagine the impracticality, if not impossibility, of taking advantage of Statistics Canada’s periodic updates of subtler, and potentially more valuable, census-based socioeconomic data.
  2. For a given year within these twenty-eight datasets, the data corresponding to the Child, Youth, LCG-Combined, CFSA-Child, and Emerging Adult classifications of age are identified with the prefix Child_, Youth_, Young_, CFSAC_, and Mixed_, respectively, plus a suffix for the year. For data relating to annual growth of population, the suffix refers to the later year in the comparison.

Land Mass of Local Health Integration Networks (LHINs) in Ontario

After spending more time than I care to admit searching for the land masses of Ontario’s fourteen Local Health Integration Networks (LHINs), I came upon this site that provided Total Population and Population Density figures, from which I derived the Land Mass:

Table 1. Land Mass of Local Health Integration Networks (LHINs) in Ontario, December 2013.
LHIN_Name LHIN_ID Total
Population
Population
Density
Land
Mass (sq km)
Erie St. Clair 3501 619055 84.53 7323.49
South West 3502 925415 44.26 20908.61
Waterloo Wellington 3503 723445 152.29 4750.44
Hamilton Niagara Haldimand Brant 3504 1358805 209.79 6476.98
Central West 3505 840050 324.19 2591.23
Mississauga Halton 3506 1109545 1052.33 1054.37
Toronto Central 3507 1150010 5984.73 192.16
Central 3508 1703350 622.77 2735.12
Central East 3509 1498645 97.39 15388.08
South East 3510 478260 26.20 18254.20
Champlain 3511 1230655 69.48 17712.36
North Simcoe Muskoka 3512 439400 52.03 8445.13
North East 3513 553090 1.40 395064.29
North West 3514 222090 0.55 403800.00

Aligning Ontario’s Scheme for Identifying Census Divisions with Canada’s

Ontario’s Ministry of Finance regularly updates its population projections for the province; its most recent updates were published in the Spring 2016. These population projections are organized into 4 different datasets:

  • projections for the whole province
  • projections for each census division
  • projections for each Local Health Integration Network (LHIN)
  • projections for each Ministry of Children and Youth Services’ Service Delivery Division (SDD) region

Unfortunately (even inexplicably), Ontario uses a different scheme for identifying Census Divisions from Canada’s. We may use this map:

Ontario 2011 Census Divisions - Statistics Canada

and this map:

MofF - Chart 00

allow us to generate the following table of alignments:

Table 4. Aligning Ontario’s scheme for identifying Census Divisions with Canada’s.
CD_ID
(Ontario)
CD_ID
(Canada)
CD_ID
(Ontario)
CD_ID
(Canada)
1 20 26 13
2 18 27 47
3 24 28 1
4 21 29 41
5 19 30 34
6 29 31 37
7 22 32 42
8 28 33 40
9 46 34 36
10 25 35 38
11 44 36 39
12 26 37 32
13 14 38 31
14 15 39 57
15 43 40 56
16 16 41 51
17 30 42 48
18 23 43 49
19 6 44 53
20 10 45 52
21 12 46 54
22 9 47 60
23 7 48 59
24 11 49 58
25 2

We will need to make use of this Table of alignments when we come to map the Ministry of Finance’s population projections onto the boundaries of Ontario’s Local Health Integration Networks (LHINs).

Disclaimer: This post is my personal work and is not sponsored or endorsed by Youthdale Treatment Centres in any way. This work  is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Population Projections in Ontario: The Case for Returning to Life Cycle Groupings

Background

Every year the Ministry of Finance updates its population projections for Ontario and each of its 49 census divisions to reflect the most recent trends and historical data. The Spring 2016 update is based on new 2015 population estimates from Statistics Canada and reflects minor changes in trends in fertility, mortality and migration.

Map of Ontario Census Divisions
Figure 1. Map of Ontario Census Divisions. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.

The Ministry of Finance includes several Charts and Tables that reference various demographic groupings in its analysis of these population projections:

Demographic Groupings
Single Years (0, 1, …, 90+) Statistical Table 6
Five-Year Groupings (0-4, 5-9, …, 85-89, 90+) Statistical Tables 7-10
Life Cycle Groupings (0-14, 15-64, 65+) Charts 5-6, 10-12
Statistical Table 2

To assist in planning, delivering, and evaluating human services – particularly those for young people – we want to differentiate Youth (15-24 years old) from Adult (25-64) instead of using the Ministry of Finance’s original definition of “Adult”: 1

  • Children (0- 14 years old)
  • Youth (15 – 24)
  • Adults (25 – 64)
  • Seniors (65+)

For the interested reader, we have compiled a set of Statistical Tables that restate the population projections for Ontario in terms of our Life Cycle Groupings.

Now, let’s review the highlights of the Ministry of Finance’s analysis.

Provincial overview

The Ministry of Finance considers three scenarios of population growth in Ontario. The medium-growth or reference scenario – the most likely to occur if recent trends continue – projects population growth of 30.1 per cent, from 13.8 million in 2015 to more than 17.9 million in 2041 (Chart 1).

Ontario population, 1971 to 2014
Chart 1. Ontario population, 1971 to 2014. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.

The rate of population growth in Ontario in the reference scenario is projected to decline gradually from 1.2 per cent to 0.8 per cent annually (Chart 2).

Annual rate of population growth in Ontario, 1971 to 2041
Chart 2. Annual rate of population growth in Ontario, 1971 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.

Components of population growth

In any given year, the share of population growth due to natural increase versus net migration varies. While natural increase trends evolve slowly, net migration can be more volatile, mostly due to swings in inter-provincial migration and variations in international immigration.

Natural increase

The number of births and deaths in Ontario has been rising slowly and at a similar pace over the last decade. As a result, natural increase has been fairly stable at about 50,000 annually. The rate of population growth due to natural increase over the projection period is affected by two main factors:

  • The passage of the baby boom echo generation (children of baby boomers) through peak fertility years will result in an increased number of births through the late 2010s and early 2020s.
  • The transition of large cohorts of baby boomers into the Seniors group.

Overall, natural increase is projected to be fairly stable around 55,000 over the first decade of the projections, followed by a steady decline to less than 17,000 by 2041. The share of population growth accounted for by natural increase (versus net migration) is projected to decline from 32 per cent in 2016 to 11 per cent by 2041 (Chart 3).

Net migration

Net migration to Ontario has averaged about 77,000 per year in the past decade. Net migration is projected to be higher at the beginning of the projection period than it has been during the past few years, as net losses of population through inter-provincial migration have recently turned to gains and federal immigration targets have been raised significantly.

Ontario’s annual net migration gain is projected to increase from 114,000 in 2016 to 130,000 by 2041. The share of population growth accounted for by net migration (versus natural increase) is projected to rise from 68 per cent in 2016 to 89 per cent by 2041 (Chart 3).

Contribution of natural increase and net migration to population growth in Ontario, 1971 to 2041.
Chart 3. Contribution of natural increase and net migration to population growth, 1971 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.

Age structure

The Ministry of Finance displays the distribution of age among the people of Ontario in the familiar form of an age pyramid (Chart 4) and shows how age structure impacts the share of population (Chart 5) and the rate of population growth (Chart 6) accounted for by three Life Cycle Groupings (0-14, 15-64, 65+ years old).

Using our four Life Cycle Groupings ((0-14, 15-24, 25-64, 65+ years old), we have redrawn the projected share of population (Chart 5-PGA) and the projected rate of population growth (Chart 6-PGA):

Age pyramid in Ontario, 2015 and 2041.
Chart 4. Age pyramid in Ontario, 2015 and 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.
Proportion of population aged 0-14, 15-64, and 65+ in Ontario, 1971 to 2041
Chart 5. Proportion of population aged 0-14, 15-64, and 65+ in Ontario, 1971 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.
Proportion of population aged 0-14, 15-24, 25-64, and 65+ in Ontario, 2016 to 2041
Chart PGA 5. Proportion of population aged 0-14, 15-24, 25-64, and 65+ in Ontario, 2016 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.
Annual rate of growth of population age groups 0-14, 15-64, and 65+ in Ontario, 1971 to 2041
Chart 6. Annual rate of growth of population age groups 0-14, 15-64, and 65+ in Ontario, 1971 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.
Annual rate of growth of population age groups 0-14, 15-24, 25-64, and 65+ in Ontario, 2017 to 2041
Chart PGA 6. Annual rate of growth of population age groups 0-14, 15-24, 25-64, and 65+ in Ontario, 2017 to 2041. Source: Ministry of Finance, Ontario Population Projections Update, Spring 2016.

The Ministry of Finance includes the following analysis (pp. 8-10):

By 2041, there will be more people in every age group in Ontario compared to 2015, with a sharp increase in the number of seniors. Baby boomers will have swelled the ranks of seniors; children of the baby boom echo generation will be of school-age; and the baby boom echo cohorts, along with a new generation of immigrants, will have bolstered the population aged 15–64. ...

The number of children aged 0–14 is projected to increase gradually over the projection period, from 2.2 million in 2015 to almost 2.7 million by 2041. The share of children in the population is projected to decrease from 15.9 per cent in 2015 to 14.9 per cent by 2041. By the late 2030s, the number of children is projected to grow at a much slower pace than other age groups, reflecting the smaller number of women in their 20s and 30s. ...

Within the 15–64 age group, the number of youth aged 15–24 is initially projected to decline slightly, from a high of 1,827,000 in 2015 to a low of 1,725,000 by 2022. The youth population is then projected to resume growing, reaching almost 2.1 million by 2041. The youth share of total population is projected to decline from 13.2 per cent in 2015 to 11.1 per cent by 2033, followed by a small rise to 11.5 per cent by 2041. ...

In this last paragraph, the Ministry of Finance makes its one and only substantial reference to “youth” – yet it’s a pretty important point: While the number of Youth in Ontario is projected to decline over the next five years, their numbers will increase steadily thereafter – in fact, becoming the fastest growing demographic by the end of the projection period. The Ministry’s graphics (Charts 5-6), unfortunately, obscure what’s going on here – as is immediately apparent when we differentiate Youth (15-24 years old) from Adult (25-64) using the Life Cycle Groupings  (Charts PGA 5-6).

Next time: We “drill down” to the level of the region and census division – where the planning, delivery and evaluation of human services take place or where, at least, I’d argue that they should. Meanwhile, the interested reader may want to consult our Statistical Tables that restate the population projections for Ontario in terms of the Life Cycle Groupings.

  1. In fact, Statistics Canada used these four Life Cycle Groupings until 2007. In this sense, we’re arguing for the return to Life Cycle Groupings when it comes to understanding population projections – especially when one is concerned with young people. The Ministry of Finance refers to “youth” only twice – once in passing while noting that some census divisions of Northern Ontario experiencing net out-migration, mostly among youth (p. 12) and once in the context of a more substantial discussion of changes in the annual rate of population growth due to age structure (p. 10, and see below).

Overview

Every year the Ministry of Finance (MoF) updates its population projections for Ontario and each of its 49 census divisions to reflect the most recent trends and historical data. The Spring 2016 update is based on new 2015 population estimates from Statistics Canada and reflects minor changes in trends in fertility, mortality and migration.

MofF - Chart 00
Figure 1. Map of Ontario Census Divisions.

The projections provide three reasonable growth scenarios for the population of Ontario to 2041. The medium-growth or reference scenario is considered most likely to occur if recent trends continue. The low- and high-growth scenarios provide a forecast range based on plausible changes in the components of growth. Population is projected for each of the 49 census divisions for the reference scenario only.

The MoF’s analysis and commentary on the population projections includes several Charts and Tables that reference various geographic units.

Geographic Units
Province Charts 1-4, 13-19; Table B; Appendix: Tables 1-3
Regions (6) Chart 8; Table A; Appendix: Tables 10-15
Census Divisions (49) Charts 7, 9-12; Appendix: Tables 4-5, 10.1-10.5, 11.1-11.13, 12.1-12.10, 13.1-13.10, 14.1-14.8, 15.1-15.3
Demographic Units
Single Years (0, 1, …, 90+) Appendix: Table 6
Five-Year Groupings (0-4, 5-9, …, 85-89, 90+) Appendix: Tables 7-10
Life Cycle Groupings (0-14, 15-64, 65+) Charts 5-6, 10-12; Appendix: Table 2

Our contribution

To assist in planning, delivering, and evaluating mental health services for children and youth, we extend the MoF’s analysis and commentary on the population projections to include the Children and Youth Mental Health (CYMH) Service Areas used by the Ministry of Children and Youth (MCYS). For the CYMH Service Areas, we derive two datasets:

We also derive different population projections for young people:

  • simple groupings of
    • 0 – 17 year olds (to align with the definition of “child” in the Child and Family Services Act (CFSA), and
    • 0 – 24 year olds (a demographic that is arguably more meaningful from a developmental point of view)
  • life cycle groupings of
    • children (0 – 14 year olds), and
    • youth (15 – 24 year olds)
  • transitional-age group of
    • young persons making the transition from mental health services for “children” (16 and 17 year olds) under the CFSA to mental health services for “emerging adults” (18, 19, and 20 year olds) under the Mental Health Act in Ontario
Census Districts (49)
   Land Area (forthcoming)
   Total Population (forthcoming)
CYMH Service Areas (33)
  Land Area CYMHSAS_Land_Area.csv
  Total Population CYMHSAS_Total_Population_2016_2041.csv
Integrated Service Regions (5)
   Land Area (forthcoming)
   Total Population (forthcoming)
Local Health Integration Networks (14)
   Land Area (forthcoming)
   Total Population (forthcoming)

We also derive different population projections for young people:

  • simple groupings of
    • 0 – 17 year olds (to align with the definition of “child” in the Child and Family Services Act that governs the work of the MCYS), and
    • 0 – 24 year olds (a demographic that is arguably more meaningful from a developmental point of view)
  • life cycle groupings of
    • 0 – 14 year olds (children), and
    • 15 – 24 year olds (youth)
  • a transitional-age group of
    • young persons making the transition from mental health services for “children” (16 and 17 years old) under the MCYS/CFSA to mental health services for “emerging adults” (18, 19, and 20 years old) under the MOHLTC/Mental Health Act in Ontario

CYMHSAS_0017_2016_2041.csv, CYMHSAS_1620_2016_2041.csv and others …., have this structure:

area_name

id

Population_YYYY, where YYYY = {2016, 2017, …, 2041}

Density_YYYY, where YYYY = {2016, 2017, …, 2041}

Share _YYYY, where YYYY = {2016, 2017, …, 2041}

AnnualGrowth_YYYY, where YYYY = {2017, 2018, …, 2041}

Growth_YYYY_vs_2017, where YYYY = {2021, 2026, 2031, 2036, 2041}

Population_1524_2016; .

Growth_1524_2036_vs_2017

..

Geographic Units
Integrated Service Regions (5) (forthcoming)
CYMH Service Areas (33)
LHINs (14) (forthcoming)
Demographic Units
Simple Groupings (0-17, 0-24) CYMHSAS_0017_2016_2041.csv
Life Cycle Groupings (0-14, 15-24) CYMHSAS_0014_2016_2041.csv

CYMHSAS_1524_2016_2041.csv

Transitional-Age Grouping (16-20) CYMHSAS_1620_2016_2041.csv

Under all three scenarios, Ontario’s population is projected to experience moderate growth over the 2015–2041 period. In the reference scenario, population is projected to grow 30.1 per cent, or almost 4.2 million, over the next 26 years, from an estimated 13.8 million on July 1, 2015 to more than 17.9 million on July 1, 2041 (Chart 1).

MofF - Chart 1
Chart 1. Ontario population, 1971 to 2014

The annual rate of growth of Ontario’s population in the reference scenario is projected to decline gradually from 1.2 per cent to 0.8 per cent over the projection period (Chart 2).

MofF - Chart 2
Chart 2. Annual rate of population growth in Ontario, 1971 to 2041.

Components of population changeIn any given year, the contributions of natural increase and net migration to population growth vary. While natural increase trends evolve slowly, net migration can be more variable, mostly due to swings in interprovincial migration and variations in international immigration.The number of births and deaths has been rising slowly and at a similar pace. As a result, natural increase has been fairly stable at about 50,000 annually over the last decade.Net migration levels to Ontario have averaged about 77,000 per year in the past decade, with a low of 52,000 in 2006–07 and a high of 96,000 in 2011–12.Net migration is projected to be higher at the beginning of the projections than it has been during the past few years as net losses of population through interprovincial migration have recently turned to gains and federal immigration targets were raised by a significant amount. Ontario’s annual net migration gain is projected to increase over the projection period from 114,000 in 2015–16 to 130,000 by 2040–41. The share of population growth accounted for by net migration is projected to rise from 68 per cent to almost 89 per cent by 2041 as a result of lower natural increase.

MofF - Chart 3
Chart 3. Contribution of natural increase and net migration to population growth, 1971 to 2041.

Future levels of natural increase will be affected by two main factors over the projection period:

  • The passage of the baby boom echo generation (children of baby boomers) through peak fertility years, which results in an increase in the number of births through the late 2010s and early 2020s.
  • The transition of large cohorts of baby boomers into the senior age group.

Overall, natural increase is projected to be fairly stable around 55,000 over the first decade of the projections, followed by a steady decline to less than 17,000 by 2040–41. The share of population growth accounted for by natural increase is projected to decline from 32 per cent in 2015–16 to 11 per cent by 2040–41.Age structureBy 2041, there will be more people in every age group in Ontario compared to 2015, with a sharp increase in the number of seniors. Baby boomers will have swelled the ranks of seniors; children of the baby boom echo generation will be of school-age; and the baby boom echo cohorts, along with a new generation of immigrants, will have bolstered the population aged 15–64.

MofF - Chart 4
Chart 4. Age pyramid of population, 2015 to 2041

The median age of Ontario’s population is projected to rise from 41 years in 2015 to 45 years in 2041. The number of seniors aged 65 and over is projected to more than double from about 2.2 million, or 16.0 per cent of population in 2015, to over 4.5 million, or 25.3 per cent, by 2041. In 2015, for the first time, seniors accounted for a larger share of population than children aged 0–14.

MofF - Chart 5
Chart 5. Proportion of population aged 0-14, 15-64, and 65+, 1971 to 2041.

The number of children aged 0–14 is projected to increase gradually over the projection period, from 2.2 million in 2015 to almost 2.7 million by 2041. The share of children in the population is projected to decrease from 15.9 per cent in 2015 to 14.9 per cent by 2041. By the late 2030s, the number of children is projected to grow at a much slower pace than other age groups, reflecting the smaller number of women in their 20s and 30s.

MofF - Chart 6
Chart 6. Pace of growth of population age groups 0-14, 51-64 and 65+, 1971 to 2041.

Regional components of population changeThe main demographic determinants of regional population growth are the current age structure of the population, the pace of natural increase, and the migratory movements in and out of each of Ontario’s regions. Demographic trends vary significantly among the 49 census divisions that comprise the six geographical regions of Ontario.The current age structure of each region has a direct impact on projected regional births and deaths. A region with a higher share of its current population in older age groups will likely experience more deaths in the future than a region of comparable size with a younger population. Similarly, a region with a large share of young adults in its population is expected to see more births than a region of comparable size with an older age structure. Also, since migration rates vary by age, the age structure of a region or census division will have an impact on the migration of its population.The general aging of the population will result in a rising number of census divisions where deaths will exceed births (negative natural increase) over the projection period. Deaths exceeded births in 24 of Ontario’s 49 census divisions over the past five years. This number is projected to rise gradually so that 37 census divisions are projected to experience negative natural increase by 2040– 41. These 37 census divisions will represent 26 per cent of Ontario’s population in 2041.This declining trend in natural increase means that many census divisions in Ontario where natural increase previously was the main or even sole contributor to population growth have already started to see their population growth slow. This trend is projected to continue as the population ages further.

MofF - Chart 7
Chart 7. Evolution of natural increase by census division, 2015 to 2041.

Migration is the most important factor contributing to population growth for Ontario as a whole and for most regions. Net migration gains, whether from international sources, other parts of Canada or other regions of Ontario, are projected to continue to be the major source of population growth for almost all census divisions.

MofF - Chart 9
Chart 8. Population growth/decline by census division over 2015 to 2041.

Large urban areas, especially the GTA, which receive most of the international migration to Ontario, are projected to grow strongly. For other regions such as Central Ontario, the continuation of migration gains from other parts of the province will be a key source of growth. Some census divisions of Northern Ontario receive only a small share of international migration and have been experiencing net out-migration, mostly among youth, which reduces both current and future population growth.Table 1. Population shares of Ontario Regions, 1991 to 2041.Regional age structureAll regions see a shift to an older age structure. Regions where natural increase and net migration are projected to become or remain negative see the largest shifts in age structure.The GTA is expected to remain the region with the youngest age structure, a result of strong international migration and positive natural increase. The Northeast is projected to remain the region with the oldest age structure.

MofF - Chart 10
Chart 8. Share of seniors in population by census division in 2041.

x

MofF - Chart 11
Chart 9. Growth in number of seniors by census division, 2015 to 2041.

The number of children aged 0–14 is projected to decline in the North, but to increase in the rest of Ontario over the projection period. However, by 2041 the share of children in every region is projected to be slightly lower than it is today. In 2015, the highest share of children among regions was in the Northwest at 16.9 per cent; the Northeast had the lowest share at 14.4 per cent. By 2041, the Northeast is projected to remain the region with the lowest share of children at 13.3 per cent while the highest share is projected to be found in the Northwest at 15.5 per cent.The suburban GTA census divisions, along with Ottawa, are projected to record the highest growth in the number of children aged 0–14 over the 2015–2041 period, with Halton seeing the most growth at 49 per cent. Conversely, the majority of rural and northern census divisions are projected to have significantly fewer children by 2041, with the largest declines in the North. However, most census divisions are projected to see only a slight decrease in the share of children in their population. In 2015, the highest share of children was found in Kenora at 21.9 per cent and the lowest share in Haliburton at 9.9 per cent. By 2041, Kenora is projected to still have the highest share of children at 20.0 per cent while Haliburton is projected to continue to have the lowest at 9.1 per cent.

MofF - Chart 12
Chart 10. Growth/decline in number of children aged 0-14 by census division, 2015 to 2041.
1 2
Population (%) 1991 2001 2011 2021 2031 2041
GTA 42.0 44.5 47.2 49.3 51.1 52.7
Central 22.2 22.1 21.6 21.2 20.8 20.5
East 13.9 13.5 13.2 12.9 12.7 12.4
Southwest 13.7 13.0 12.0 11.2 10.5 9.9
Northeast 5.8 4.8 4.3 3.8 3.3 3.0
Northwest 2.4 2.1 1.8 1.6 1.5 1.3

Population Projections across CYMH Service Areas, 2016-2041

Population Projections

Recently the Ministry of Finance updated its population projections for Ontario, 2016 – 2041. These population projections are organized into 4 different datasets:

  • projections for the whole province
  • projections for each census division
  • projections for each Local Health Integration Network (LHIN)
  • projections for each Ministry of Children and Youth Services’ Service Delivery Division (SDD) region

Each dataset includes population projections by age and gender.

From the population projections for each census division (CD), we have derived the population projections (summing projections for Males and Females) for each of the CYMH Service Areas, 2016 – 2041.

The result of our work is presented in one spreadsheet, that includes six worksheets:

  • Ages x Year (Base): Population projections for each Age between 0 – 24 years by Year
  • 0-18 x Year: Population projections for the total number of 0 – 18 year old children and youth by Year
  • 0-24 x Year: Population projections for the total number of 0 – 24 year old children and youth by Year
  • 0-18 x Age Group x Year: Population projections for children and youth grouped into 5 year spans (0-4, 5-9, 10-14, and 15-18 years old)
  • 0-24 x Age Group x Year: Population projections for children and youth grouped into 5 year spans (0-4, 5-9, 10-14, 15-19, and 20-24 years old)
  • 0-24 x Life Cycle x Year: Population projections for children and youth grouped into two Life Cycle spans (Child, 0 – 14 years old and Youth, 15 – 24 years old)

Population Density

We have also calculated the land area of the individual CYMH Service Areas in order to derive their respective projected population densities. We will soon provide a speadsheet with these projections as well.

For now, let’s use the worksheet containing the projections of population and population densities for 0 – 18 year olds to add some (non-geospatial) data finally to our visualization of the CYMH Service Areas. Our illustration of two mapping techniques – proportional symbol representation (for the population projections) and choropleth (for the projected population densities) – will be illustrative only.

Proportional Symbol Representation of Population Projections

We may represent the projected number of 0 – 18 year olds across all CYMH Service Areas in a given year by combining geospatial data (in the familiar TopoJSON file format) with demographic data (in .csv format):

Proportional Symbol representation of projected population of 0-18 year olds screenshot
Figure 1. Proportional symbol representation of projected population of 0 – 18 year olds across the CYMH Service Areas in 2020.

[Interactive page – try hovering the mouse over a bubble]

Choropleth Representation of Projected Population Densities

We may also represent the projected density of 0 – 18 year olds across all CYMH Service Areas in a given year by combining geospatial data (in the familiar TopoJSON file format) with demographic data (in .csv format):

Choropleth Population Density 0-18 year olds screenshot
Figure 2. Choropleth representation of projected population density of 0 – 18 year olds across the CYMH Service Areas in 2020.

[Interactive page – try hovering the mouse over a CYMH Service Area]

Next time: We’ll look at increasing user interaction with choropleths and proportional symbol representations – including animation!

… and then there were 33

In a series of previous posts, I have been exploring the use of D3 (Data Driven Documents) – a Free and Open Source Software package – to visualize geo-spatial data associated with the Children and Youth Mental Health (CYMH) Service Areas that have been established recently by the Ministry of Children and Youth Services (MCYS) in Ontario. To avoid confusion, I wanted to alert users of some of the resources that I have published of an important development.

The MCYS has been resourcing the administration and functions of the CYMH Service Areas, including the designation of Lead Agencies, over the past few years. During this time, there has been some uncertainty about whether there were to be thirty-three or thirty-four CYMH Service Areas – turning on whether the James Bay Coast would be its own Service Area or would be merged with the Timiskaming/Cochrane Service Area.

When I began to explore the use of d3 to visualize the CYMH Service Areas, the geo-spatial data published by the Ontario government mapped thirty-four Service Areas (e.g. see the Wayback Machine archive of September 6, 2015!) and the resources that I have published reflected this configuration. Now the geo-spatial data published by the government maps only thirty-three CYMH Service Areas.

I’ve completed the revision of resources for users in the past few days – a slight inconvenience for us all. There is a bigger issue, though:

The Ontario government is providing an incredibly valuable resource when it publishes the geo-spatial data associated with the administration of public services, like children and youth mental health services. I would only urge that the government’s web pages that describe and make geo-spatial resources available to us should retain and present the different versions of these resources over time. This approach would not only avoid possible confusion as revisions are made, but the differences between the versions may themselves be of interest to the public.

CYMHSAS_33

Access geospatial data from MCYS

The Ministry of Children and Youth Services (MCYS) has defined two administrative views of Ontario’s geography:

  • Integrated Service Regions
  • Children and Youth Mental Health Service Areas

Children and Youth Mental Health (CYMH) Service Areas

The Ministry of Children and Youth Services (MCYS) funds a variety of planned, multidisciplinary interventions for children, youth and their families in thirty-three Service Areas across Ontario. 1 Let’s download the shapefile that contains the geospatial data used by the provincial government to map these Service Areas and rename the shapefiles “CYMH-Service-Areas.*”.

Generate GeoJSON files

We use ogr2ogr to convert the shapefiles to GeoJSON files:


ogr2ogr -t_srs EPSG:4269 -f GeoJSON cymhsas_geo.json CYMH-Service-Areas.shp

We use Notepad++ to give more meaningful names to the variables in cymhsas_geo.json in all lowercase letters:

cymhsas_geo.json
Original variable Modified variable
ServiceAre area_index
ServiceA00 area_name

Next: We convert the geojson file into a more compact topojson file:

topojson -o cymhsas_topo.json --id-property area_index  --properties -- cymhsas_geo.json

xx

Age Categories and Life Cycle Groupings – Statistics Canada

Policy on standards (revised July 14, 2004)

Introduction

Statistics Canada aims to ensure that the information it produces provides a consistent and coherent picture of the Canadian economy, society and environment, and that its various datasets can be analyzed together and in combination with information from other sources.

To this end, the Agency pursues three strategic goals:

  1. The use of conceptual frameworks, such as the System of National Accounts, that provide a basis for consolidating statistical information about certain sectors or dimensions of the Canadian scene;
  2. The use of standard names and definitions for populations, statistical units, concepts, variables and classifications in statistical programs;
  3. The use of consistent collection and processing methods for the production of statistical data across surveys.

This Policy deals with the second of these strategic goals. It provides a framework for reviewing, documenting, authorizing, and monitoring the use of standard names and definitions for populations, statistical units, concepts, variables and classifications used in Statistics Canada’s programs. Standards for specific subject-matter areas will be issued from time to time under this Policy as required.

Policy

Statistics Canada aims to use consistent names and definitions for populations, statistical units, concepts, variables, and classifications used in its statistical programs. To this end:

  1. Statistical products will be accompanied by, or make explicit reference to, readily accessible documentation on the definitions of populations, statistical units, concepts, variables and classifications used.
  2. Wherever inconsistencies or ambiguities in name or definition are recognized between related statistical units, concepts, variables or classifications, within or across programs, the Agency will work towards the development of a standard for the statistical units, concepts, variables and classifications that harmonize such differences.
  3. Standards and guidelines covering particular subject-matter areas will be issued from time to time and their use will be governed by the provisions of this Policy.
  4. Where departmental standards have been issued, program areas must follow them unless a specific exemption has been obtained under the provisions of this Policy.
  5. Programs should, to the extent possible, collect and retain information at the fundamental or most detailed level of each standard classification in order to provide maximum flexibility in aggregation and facilitate retrospective reclassification as needs change.
  6. When a program uses a population, statistical unit, concept, variable or classification not covered by a departmental standard, or uses a variation of a standard approved as an exemption, it shall use a unique name for the entity to distinguish it from any previously defined standard.
  7. Clients of Statistics Canada’s consultative services should be made aware of and encouraged to conform to the standards and guidelines issued under this Policy.
  8. The Agency will build up a database of names and definitions used in its programs and make this database accessible to users and other players in the statistical system.

Scope

This policy applies to disseminated data however collected, derived or assembled, and irrespective of the medium of dissemination or the source of funding. This policy may also be applied to data at the stage of collection and processing at Statistics Canada.

Guidelines for the development and documentation of standards

A. Introduction

These guidelines describe the requirements and give guidance for the development and documentation of standard names and definitions of populations, statistical units, concepts, variables and classifications. Section B defines the terminology; guidelines follow in Section C.

B. Terminology

For purposes of these guidelines the following terms are used.

Population: The set of statistical units to which a dataset refers.

Concept: A general or abstract idea that expresses the social and/or economic phenomenon to be measured.

Statistical unit: The unit of observation or measurement for which data are collected or derived. The following list provides examples of standard statistical units that have been defined.

Person
Census family
Economic family
Household
Dwelling
Location
Establishment
Company
Enterprise

Variable: A variable consists of two components, a statistical unit and a property. A property is a characteristic or attribute of the statistical unit.

Classification: A classification is a systematic grouping of the values that a variable can take comprising mutually exclusive classes, covering the full set of values, and often providing a hierarchical structure for aggregating data. More than one classification can be used to represent data for a given variable.

Example: The following is an example of the variable: Age of Person.

Concept:  Based on the subjects used by Statistics Canada to organize its statistical products and metadata, the variable Age of Person is listed under the concept of Population and Demography.

Statistical unit and property: The statistical unit and property that define this variable are Person and Age respectively. Person refers to an individual – this is the unit of analysis for most social statistics programmes. Age refers to the age of a person (or subject) of interest at last birthday (or relative to a specified, well-defined reference date).

Classification:  Different classifications can be used to represent data for this variable. These classifications include: Age Categories, Five-year Age Groups; and Age Categories, Life Cycle Groupings.

The standard names and definitions of populations, statistical units, concepts, variables and classifications will be stored in the Integrated Metadatabase (IMDB). In the case of variables, the name stored in the IMDB will include a representation type, in addition to the statistical unit and property. In the age example given here, the full name of the variable in the IMDB would be Category of Age of Person. The representation type Category indicates that it is a categorical variable, which will be represented by a classification of age groups.

C. Guidelines

Each standard should have the following characteristics:

  • describe the concept that the standard addresses when appropriate;
  • identify the statistical unit(s) to which it applies;
  • provide a name and definition of each variable included in the standard;
  • provide the classification(s) to be used in the compilation and dissemination of data on each variable.

The most detailed level of a classification will always be included in a standard. Recommended and optional aggregation structures may also be present.

Concepts shall be described in relation to a framework when possible.

Every variable shall be given a name, in both official languages, which, once given, cannot be used to denote any other variable. Variables shall be defined with explanatory notes in terms of a property and the statistical unit to which it applies. Additionally, in the IMDB, the representation type will be defined.

Every classification shall be given a name, in both official languages, which, once given, cannot be used to denote any other classification. Classifications shall be defined, with exclusions listed and explanatory notes given, where required.

Every class shall be given a name, in both official languages, which, once given, cannot be used to denote any other grouping for the referenced variable within a given “family” of classifications (i.e. a given classification and all its variants). Classes shall be defined, with exclusions listed and explanatory notes given, where required.

The most frequently used populations shall be given a name, in both official languages, which, once given, cannot be used to denote any other population. These populations shall be defined with explanatory notes.

Every statistical unit shall be given a name, in both official languages, which, once given, cannot be used to denote any other statistical unit. Statistical units shall be defined with explanatory notes.

A standard shall be accompanied by a statement of conformity to relevant internationally recognized standards, or a description of the deviations from such a standard and, where possible, a concordance with the referenced standard.

Where a standard replaces an earlier one, a concordance between the old and the new shall be given.

A standard shall include a statement regarding the degree to which its application is compulsory. The different degrees are, in descending order of compulsion:

  • departmental standard: a standard that has been approved by the Policy Committee, and the application of which is therefore compulsory, unless an exemption has been explicitly obtained under the terms of this policy;
  • recommended standard: a standard that has been recognized by the Methods and Standards Committee as a recommended standard, with or without a trial period of a specified duration, after which it may be declared as a departmental standard;
  • program-specific standard: a standard adopted by a statistical program, and which is registered with Standards Division, to ensure consistency in a series over time periods.

Status

Age of person was approved as a departmental standard on May 22, 2007.

Definition

Age refers to the age of a person (or subject) of interest at last birthday (or relative to a specified, well-defined reference date).

Person refers to an individual and is the unit of analysis for most social statistics programmes.

Derivation

Age of person is usually derived. It is usually calculated using the person’s date of birth and the date of interview or other well-defined reference date.

Relation to previous standard

A classification by single years of age has been added. In the classification of five year age groups, the top five categories in the previous classification have been collapsed into one category. These top categories were collapsed to reflect the population numbers in these categories and the reliability of the data in this part of the age range. The classification Age by life cycle groupings, which was part of the previous standard, is no longer recognized as part of the standard for age.

Conformity to relevant internationally recognized standards

This standard conforms to the recommendations for censuses contained in the United Nations’ Principles and Recommendations for Population and Housing Censuses, Revision 2, 2008. The UN recommendations define age as “the interval of time between the date of birth and the date of the census, expressed in completed solar years”. This is equivalent to this standard’s definition of age as “age at last birthday”. In addition, the UN recommends calculating age from date of birth rather than asking it directly. This derivation of age is recognized in this standard as the usual practice. Use of date of birth, as noted in the UN Principles, allows age to be calculated precisely, avoiding rounding by respondents and potential misunderstanding as to whether the age wanted is that of the last birthday, the next birthday or the nearest birthday. Finally, in the suggested census output tables, the UN Principles use five-year age groupings with the same boundaries as those presented in this standard. The only differences from this standard are that the upper category has a lower boundary (typically, “85 and over”) and that sometimes children under age 1 year are reported in a separate category.

The Conference of European Statisticians Recommendations for the 2010 Censuses of Population and Housing also recommends that information on age be obtained by collecting information on date of birth.

 This classification was replaced by a new departmental standard on May 22, 2007.
 ID  Age Range
10 0-4 years
11 5-9 years
12 10-14 years
13 15-19 years
14 20-24 years
15 25-29 years
16 30-34 years
17 35-39 years
18 40-44 years
19 45-49 years
20 50-54 years
21 55-59 years
22 60-64 years
23 65-69 years
24 70-74 years
25 75-79 years
26 80-84 years
27 85-89 years
28 90-94 years
29 95-99 years
30 100-104 years
31 105-109 years
32 110-114 years
33 115-119 years
34 120-124 years

Age Categories, Life Cycle Groupings

1 Children (00-14 years)
11 00-04 years
110 00-04 years
12 05-09 years
120 05-09 years
13 10-14 years
130 10-14 years
2 Youth (15-24 years)
21 15-19 years
211 15-17 years
212 18-19 years
22 20-24 years
221 20-21 years
222 22-24 years
3 Adults (25-64 years)
31 25-29 years
310 25-29 years
32 30-34 years
320 30-34 years
33 35-39 years
330 35-39 years
34 40-44 years
340 40-44 years
35 45-49 years
350 45-49 years
36 50-54 years
360 50-54 years
37 55-59 years
370 55-59 years
38 60-64 years
380 60-64 years
4 Seniors (65 years and over)
41 65-69 years
410 65-69 years
42 70-74 years
420 70-74 years
43 75-79 years
430 75-79 years
44 80-84 years
440 80-84 years
45 85-89 years
450 85-89 years
46 90 years and over
460 90 years and over

Overlaying the boundaries of the Local Health Integration Networks and Children and Youth Mental Health Service Areas in Ontario

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In a series of previous posts, we have visualized the boundaries of the MCYS Integrated Service Regions (ISRs) and the MCYS Children and Youth Mental Health (CYMH) Service Areas.

Now we want to merge the digital boundaries of the CYMH Service Areas with the MOHLTC’s Local Health Integration Networks (LHINs) in Ontario.

The boundaries of the LHINs in 2015 are provided in the ESRI ® shapefile format (HRL035b11a_e.zip) by Statistics Canada. The .zip file contains four familiar files:

  • HRL03b11a_e.dbf
  • HRL03b11a_e.prj
  • HRL03b11a_e.shp
  • HRL03b11a_e.shx

The projection information in the HRL03b11a_e.prj file indicates that the geospatial data for the LHINs uses the EPSG 3347 PCS Lambert Conformal Conic projection. 1 So we convert the original shapefile for the LHINs to the same projection (EPSG 4269) used for the CYMH Service Areas:

ogr2ogr -f 'ESRI Shapefile' -t_srs EPSG:4269 lhins.shp HRL035a11a_e_Sept2015.shp

Next we convert the new shapefile lhins.shp to a GeoJSON file and then to a TopoJSON file (see Visualizing the MCYS Integrated Service Regions Using d3.geo for the details of this process):

ogr2ogr -t_srs EPSG:4269 -f GeoJSON lhins_geo.json lhins.shp
topojson -o lhins_topo.json --properties -- lhins_geo.json

Finally, to merge the TopoJSON file for the CYMH Service Areas and the TopoJSON file for the LHINs into a single TopoJSON file, we need to install the utility geojson-merge:

npm install -g geojson-merge

and then run:

geojson-merge cymhsas_geo.json lhins_geo.json > cymhsas_lhins_geo.json

The properties of cymhsas_lhins_topo.json include:

HR_UID -> idHealth Region ID

Property Value
area_index Identifier of CYMH Service Area
area_name Name of CYMH Service Area
ENG_LABEL -> lhin_name Name of LHIN (English)
FRE_LABEL Name of LHIN (French)

We use Notepad++ to rename ENG_LABEL to lhin_name. We then promote HR_UID to the id property of the TopoJSON file.

topojson -o cymhsas_lhins_topo.json --id-property HR_UID --properties -- cymhsas_lhins_geo.json

And finally, we use Notepad++ to add the isr and color properties to the CYMH Service Areas.

Notes:

Our visualization of the MCYS and MOHLTC geospatial data includes the following features:

  • the five Integrated Service Regions and their respective CYMH Service Areas are distinguished with different hues
  • the boundary and name of a Local Health Integrated Network are displayed when the user hovers the mouse over one of the thirteen LHINs
  • the user may pan and zoom in on the visualization

Our visualization requires only these few modifications of the Javascript we developed to display the names of the CYMH Service Areas:

/* CSS */
...
.lhin_area:hover{
  stroke: #000;
  stroke-width: 1.5 px;
}

/* Javascript */
...

function draw(topo) {

var service_area = g.selectAll(".area_name").data(topo);

/* Visualize the CYMH Service Areas in colour, use id index to assign transparent colour to LHINs */
service_area.enter().insert("path")
.attr("class", "lhin_area")
.attr("d", path)
.attr("id", function(d,i) { return d.id; })
.style("fill", function(d,i) { return i <= 32 ? d.properties.color : 'transparent' });

/* define offsets for displaying the tooltips */
var offsetL = document.getElementById('map').offsetLeft+20;
var offsetT = document.getElementById('map').offsetTop+10;

/* toggle display of tooltips in response to user mouse behaviours*/
service_area

.on("mousemove", function(d,i) {
var mouse = d3.mouse(svg.node()).map( function(d) { return parseInt(d); } );
tooltip.classed("hidden", false)
.attr("style", "left:"+(mouse[0]+offsetL)+"px;top:"+(mouse[1]+offsetT)+"px")
.html(d.properties.lhin_name);
})

.on("mouseout", function(d,i) {
tooltip.classed("hidden", true);
});

}

… yielding the following visualization (interactive version):

Overlay LHINs on CYMH Service Areas
Figure 1. Screenshot of LHIN superimposed on CYMH Service Areas.

Next time: We’ll begin to merge the geospatial data in our TopoJSON files with demographic data in the public domain.

 

  1. Using Prj2EPSG.

Health Regions in Ontario – Boundaries and Correspondence with Census Geography

This issue describes in detail the health region limits as of December 2015 and their correspondence with the 2011 and 2006 Census geography. Health regions are defined by the provinces and represent administrative areas or regions of interest to health authorities. This product contains correspondence files (linking health regions to census geographic codes) and digital boundary files. User documentation provides an overview of health regions, sources, methods, limitations and product description (file format and layout).

This issue contains the health region limits as of December 2015 and their correspondence with 2011 Census geography.

The boundaries, health region codes and health region names in Ontario have not changed.

Appendices and tables

Background

In recent years there has been an increasing demand for relevant health information at a ‘community’ level. As a result, health regions have become an important geographic unit by which health and health-related data are produced.

Health regions are legislated administrative areas defined by provincial ministries of health. These administrative areas represent geographic areas of responsibility for hospital boards or regional health authorities. Health regions, being provincial administrative areas, are subject to change.

The 2015 Health Regions: Boundaries and Correspondence with Census Geography reflects the boundaries as of December 2015 and provides the geographic linkage to 2011 and 2006 Censuses.

Description

The generic term “health region” applies to a variety of administrative areas across Canada that are defined by provincial ministries of health. To complete the Canadian coverage, each northern territory is represented as health region.

The following table describes the health regions, by province, with reference to the provincial legislation under which these areas have been defined.

Health region code structure

A four digit numeric code is used to uniquely identify health regions. The first two digits represent the province, and the second two digits represent the health region. These codes reflect the same codes used by the provincial ministries of health. For those provinces where a numeric code is not applicable, a two-digit code was assigned. Ontario uses a 4-digit code for public health units. This code was truncated to the last two digits for consistency in the national health region code structure. Since Ontario has two sets of health regions, which do not entirely relate hierarchically, their codes are unique within the province.

The names of the health regions also represent the official names used by the provinces.

See Appendix 1 Health regions in Canada, 2015 (names and codes).

Correspondence files

Production of health region level data requires geographic coding tools. Since census geography does not recognize provincial health region boundaries, a health region-to-census geography correspondence file provides the linkage between health regions and their component census geographic units. These correspondence files use the smallest relevant census geographic unit.

To accommodate various data sources producing health region level data, linkage has been created for both 2011 and 2006 Census geographies. The layout of these correspondence files includes the seven-digit Standard geographic classification (SGC) code. The SGC code uniquely represents census subdivisions (CSD).

Most health regions comprise entire CSDs (see Table 2). However, there are some cases where health regions do not conform to municipalities. The 2006 Census linkage was created at the dissemination area (DA) level and block level for British Columbia, Alberta, Saskatchewan, Manitoba, and Ontario (LHINs). Even these smaller geographic areas (DA/blocks) sometimes straddle health region boundaries. In those cases, the entire DA (or block) was assigned, in conjunction with the affected province, to just one health region and therefore represents a ‘best fit’ with census geography.

Other data sources use postal codes to geographically reference data records. These data are first converted to census geographic units using the Statistics Canada postal code conversion file, and then linked to health regions based on the correspondence file.

The dissemination area/block-to-health region (DA/block-to-HR) correspondence files provided in this publication are available in CSV format.

Record layout

The record layout of the files is shown in the following tables.

http://www.statcan.gc.ca/pub/82-402-x/2015002/t/tbl03-eng.htm
Variable name Comments
DBUID2011 Uniquely identifies a dissemination block (composed of the 2-digit province or territory unique identifier followed by the 2-digit census division code, the 4-digit dissemination area code and the 2-digit dissemination block code)
CSDUID2011 Uniquely identifies a census subdivision (composed of 2-digit province or territory unique identifier followed by the 2-digit census division code and 3-digit census subdivision code)
HRUID2015 Uniquely identifies a health region (composed of 2-digit province or territory unique identifier followed by the 2-digit health region code)
HRNAME_ENGLISH Health region name, English
HRNAME_FRENCH Health region name, French
DBPOP2011 2011 Census dissemination block population

Health regions and standard geography

For the most part, health regions can be described as groupings of counties (census divisions) or municipalities (census subdivisions). This description holds especially true in the Atlantic provinces, Quebec, and Ontario (with minor exceptions in northern Ontario). In the western provinces, health regions are less likely to follow census division or census subdivision boundaries.

The following table provides a count, by province, of census subdivisions that fall in more than one health region.

Table summary
This table displays the results of Census subdivisions linked to more than one health region. The information is grouped by Provinces with splits (appearing as row headers), 2006 Census subdivisions and 2011 Census subdivisions (appearing as column headers).
Provinces with splits 2006 Census subdivisions 2011 Census subdivisions
Nova Scotia – District Health Authorities 1 0
Ontario – Local Health Integration Networks 9 11
Ontario – Public Health Units 1 4
Manitoba 7 6
Saskatchewan 45 46
Alberta 9 6
British Columbia 6 20

Census subdivisions Health region codes Health region names Population % population split in census subdivisions
Ontario – Local Health Integration Network
3519028 3505 Central West 30,476 10.6
3508 Central 257,825 89.4
Subtotal 288,301 100
3520005 3505 Central West 130,193 5
3506 Mississauga Halton 109,344 4.2
3507 Toronto Central 1,149,993 44
3508 Central 631,372 24.1
3509 Central East 594,158 22.7
Subtotal 2,615,060 100
3521005 3505 Central West 39,123 5.5
3506 Mississauga Halton 674,320 94.5
Subtotal 713,443 100
3521024 3505 Central West 59,460 100
3508 Central 0 0
Subtotal 59,460 100
3528052 3502 South West 13,416 21.2
3504 Hamilton Niagara Haldimand Brant 49,759 78.8
Subtotal 63,175 100
3542004 3502 South West 11,487 93.5
3503 Waterloo Wellington 799 6.5
Subtotal 12,286 100
3542015 3502 South West 6,871 72.2
3512 North Simcoe Muskoka 2,649 27.8
Subtotal 9,520 100
3542045 3502 South West 3,866 59.9
3512 North Simcoe Muskoka 2,587 40.1
Subtotal 6,453 100
3543003 3508 Central 10,564 99.6
3512 North Simcoe Muskoka 39 0.4
Subtotal 10,603 100
3543021 3508 Central 1,063 5.7
3512 North Simcoe Muskoka 17,442 94.3
Subtotal 18,505 100
3560090 3513 North East 0 0
3514 North West 7,031 100
Subtotal 7,031 100

Boundary files

The health region boundaries provided in this product are based on 2011 Census geographic units. The smallest geographic unit available has been used as the building block to define health regions. In general, the legislated limits respect these units, but they do not respect DAs or blocks once the legislated boundaries are digitized. In all provinces except British Columbia, Alberta, Saskatchewan, Manitoba and Ontario (LHINs), the dissemination area was used to define health regions. However, in several instances, the actual physical legal limits split DAs. In the Prairie provinces and B.C. the dissemination block (DB) was used to improve the accuracy of these boundaries. Even with this, the physical legal boundaries do not always reflect the legislated limits recognized by the provinces thus creating many instances of split dissemination blocks.

The limits that did not respect STC geometry (the splits) were digitized by utilizing maps, spatial layers and/or descriptions supplied by and with the cooperation of the authority for each province.

Method used to create health region 2015 boundary files

All processes and procedures to update the digital boundary files were carried out using ESRI Inc.® ArcGIS TM 10.2.2, Safe Software Inc. FME ® Desktop 2015, Pitney Bowes Software Inc.® MapInfo 11.5.1, Microsoft ® Access 2007, and Microsoft ® Excel 2007.

Boundary file formats

All digital health region boundaries in this publication are available in two formats: An ESRI ® shapefile format and MapInfo® table format. We’ll be using the ESRI shapefile, which is supplied in a zip file. This file expands to provide four files of different extensions which are: (DBF, SHP, PRJ and SHX). Boundary files are provided as a national boundary file and are provided as individual provincial boundary files.

Projection information

The disseminated projection coordinate system of the health region boundary files is as follows:

  • Lambert Conformal Conic
  • Datum = NAD83
  • Units = meters
  • Spheroid = GRS 1980
  • Parameters:
    • 1st standard parallel: 49° 00′ 00″
    • 2nd standard parallel: 77° 00′ 00”
    • Central Meridian: -91° 52′ 00”
    • Latitude of Projection Origin: 63° 23′ 26.43”
    • False Easting: 6200000
    • False Northing: 3000000

“Health region” refers to administrative areas defined by the provincial ministries of health.

See Table 6 Health regions in Canada – by province and territory
See Map 14 Health Regions and Peer Groups in Canada, 2015

Health region boundary changes

See the following tables for history of changes since 2000:

Health region peer groups

In order to effectively compare health regions with similar socio–economic characteristics, health regions have been grouped into ‘peer groups’. Statistics Canada used a statistical method to achieve maximum statistical differentiation between health regions. Twenty–four variables were chosen to cover as many of the social and economic determinants of health as possible, using data collected at the health region level mostly from the Census of Canada. Concepts covered include:

  • basic demographics (for example, population change and demographic structure),
  • living conditions (for example, socio-economic characteristics, housing, and income inequality), and
  • working conditions (for example, labour market conditions).

Peer groups based on 2015 health region boundaries and 2011 Census of Population and 2011 National Household Survey data are available. There are currently nine peer groups identified by letters A through I.  There have been no changes made to peer group assignments since 2014.

See Table 8 Health regions 2015 by peer group
See Table 9 Summary table of peer groups and principal characteristics

A more detailed discussion on the rationale and methods involved in the development of peer groups is available in Health Region (2014) Peer Groups – Working paper.

Health region boundary files

Digital boundary files reflecting health region limits in effect as of December 2015.

Boundary files (documentation)

ArcInfo

ARCINFO COMPREHENSIVE DIGITAL BOUNDARY FILES

Correspondence files

Code-to-code correspondence between health regions and 2011 and 2006 Census geographic units.

Correspondence files (documentation)

2011

Health region–to–2011 Census dissemination blocks for Ontario available in CSV format via a zipped file.

2011 Comprehensive Correspondence files Download

All Canada Correspondence files Download

2006

Health region–to–2006 Census dissemination area (blocks for Ontario in CSV format).

2006 Comprehensive Correspondence files Download

All Canada Correspondence files Download

Reference maps

Health regions and peer groups

This series of reference maps show the boundaries, names and codes of health regions and peer groups in Canada, by province.

About the maps

2014 reference maps
2013 reference maps
2011 reference maps (from issue 2011001 of 82-583-X)
2007 reference maps (from issue 2010001 of 82-583-X)

Adding pan and zoom to a visualization of the CYMH Service Areas in Ontario

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In previous posts, we have described:

  1. method for partitioning the Ontario government’s Shapefile archive of the thirty-four thirty-three MCYS Children and Youth Mental Health (CYMH) Service Areas into five groupings, corresponding to the MCYS Integrated Service Regions (ISRs)
  2. a method for using the TopoJSON files to visualize the CYMH Service Areas within the separate ISRs
  3. the addition of tooltips to display the names of the CYMH Service Areas across Ontario
  4. the addition of a responsive framework to ensure that our visualizations accommodate to the display capabilities of various PCs, laptops, tablets, and smart phones

Here we highlight the additional code that’s required to allow the user to pan and zoom the visualization of the CYMH Service Areas with tooltips (#3  above):

<!DOCTYPE html>
<meta charset="utf-8">
<style>

/* CSS goes here */

/* define the container element for our map */
#map {
 margin:5% 5%;
 border:2px solid #000;
 border-radius: 5px;
 height:100%;
 overflow:hidden;
 background: #FFF;
}

/* style of the text box containing the tooltip */

div.tooltip {
 color: #222; 
 background: #fff; 
 padding: .5em; 
 text-shadow: #f5f5f5 0 1px 0;
 border-radius: 2px; 
 box-shadow: 0px 0px 2px 0px #a6a6a6; 
 opacity: 0.9; 
 position: absolute;
}

/* style of the text displayed in text box of the tooltip when mouse is hovering over a CYMH Service Area */

.service_area:hover{ stroke: #fff; stroke-width: 1.5px; }

.text{ font-size:10px; }

/* otherwise, the text box of the tooltip is hidden */
.hidden { 
 display: none; 
}

</style>
<body>

/* create the container element #map */
<div id="map"></div>

/* load Javascript libraries for D3 and TopoJSON */
<script src="//d3js.org/d3.v3.min.js"></script>
<script src= "//d3js.org/topojson.v1.min.js"></script>

<script> // begin Javascript for visualizing the geo data

/* define some global variables */

var topo, projection, path, svg, g;

/* 1. Set the width and height (in pixels) based on the offsetWidth property of the container element #map */

var width = document.getElementById('map').offsetWidth;
var height = width / 2;

/* Call function setup() to create empty root SVG element with width, height of #map */

setup(width,height);

function setup(width,height){

/* 2. Create an empty root SVG element */
d3.behavior.zoom(), constructs a zoom behavior that creates an even listener to handle zoom gestures (mouse and touch) on the SVG elements you apply the zoom behavior onto

svg = d3.select('#map').append('svg')
 .style('height', height + 'px')
 .style('width', width + 'px')
.append('g') 
.call(zoom);

g = svg.append('g')
    .on("click", click);

} // end setup()

/* 3. Define Unit Projection using Albers equal-area conic projection */

var projection = d3.geo.albers()
 .scale(1)
 .translate([0,0]);

/* 4. Define the path generator - to format the projected 2D geometry for SVG */

var path = d3.geo.path()
 .projection(projection);

/* 5.0 Start function d3.json() */
/* 5.1 Load the TopoJSON data file */

d3.json("http://cartoserve.com/maps/ontario/cymhsas33_data/cymhsas_topo.json", function(error, cymhsas_topo) {
if (error) return console.error(error);

/* 5.2 Convert the TopoJSON data back to GeoJSON format */
/* and render the map using Unit Projection */

var areas_var = topojson.feature(cymhsas_topo, cymhsas_topo.objects.cymhsas_geo);

/* 5.2.1 Calculate new values for scale and translate using bounding box of the service areas */
 
var b = path.bounds(areas_var);
var s = .95 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0][1]) / height);
var t = [(width - s * (b[1][0] + b[0][0])) / 2, (height - s * (b[1][1] + b[0][1])) / 2];

/* 5.2.2 New projection, using new values for scale and translate */
projection
 .scale(s)
 .translate(t);

/* redefine areas_var in terms of the .features array, assign array to topo */
var areas_var = topojson.feature(cymhsas_topo, cymhsas_topo.objects.cymhsas_geo).features;

topo = areas_var;

/* make the map by calling our draw() function initially within the d3.json callback function */
 
draw(topo);

}); // end function d3.json

function draw(topo) {

var service_area = g.selectAll(".area_name").data(topo);

service_area.enter().insert("path")
.attr("class", "service_area")
.attr("d", path)
.attr("id", function(d,i) { return d.id; })
.style("fill", function(d, i) { return d.properties.color; })
.attr("title", function(d,i) { return d.properties.area_name; });

/* define offsets for displaying the tooltips */
var offsetL = document.getElementById('map').offsetLeft+20;
var offsetT = document.getElementById('map').offsetTop+10;

/* toggle display of tooltips in response to user mouse behaviours*/
service_area
// begin mousemove
.on("mousemove", function(d,i) {
var mouse = d3.mouse(svg.node()).map( function(d) { return parseInt(d); } );
tooltip.classed("hidden", false)
.attr("style", "left:"+(mouse[0]+offsetL)+"px;top:"+(mouse[1]+offsetT)+"px")
.html(d.properties.area_name);
}) // end mousemove
// begin mouseout
.on("mouseout", function(d,i) {
tooltip.classed("hidden", true);
}); // end mouseout

} // end draw()

function move() {

 var t = d3.event.translate;
 var s = d3.event.scale; 
 zscale = s;
 var h = height/4;

 t[0] = Math.min(
 (width/height) * (s - 1), 
 Math.max( width * (1 - s), t[0] )
 );

 t[1] = Math.min(
 h * (s - 1) + h * s, 
 Math.max(height * (1 - s) - h * s, t[1])
 );

 zoom.translate(t);
 g.attr("transform", "translate(" + t + ")scale(" + s + ")");

 //adjust the Service Area hover stroke width based on zoom level
 d3.selectAll(".service_area").style("stroke-width", 1.5 / s);

}

/* our function click() uses the .invert() configuration method */
/* to project backward from Cartesian coordinates (in pixels) to spherical coordinates (in degrees) */

function click() {
 var latlon = projection.invert(d3.mouse(this));
 console.log(latlon);
}

</script> // end Javascript for visualizing the geo data

Giving us the following interactive visualization of the CYMH Service Areas in Ontario.

Next time: We will show how to merge our visualization of the geography of the CYMH Service Areas with other data about the populations and service providers within these Service Areas.

A responsive framework for visualizing CYMH Service Areas in Ontario

In previous posts (1, 2, 3) we described a method for visualizing geographic representations of Integrated Service Regions (ISRs) and Children and Youth Mental Health Service Areas (CYMHSAs) in Ontario using free and open source software.

This brief note is to advise we are now running D3 in tandem with Bootstrap – a responsive framework that ensures that our visualizations accommodate to the display capabilities of a wide range of digital devices, including PCs, laptops, tablets, and smartphones. 1

 

  1. See our guidelines for installing Bootstrap on a Linux server.

Consolidating and enhancing the visualization of CYMH Service Areas in Ontario

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In previous posts, we provided the following:

  • method for partitioning the Ontario government’s Shapefile archive of the thirty-four MCYS Children and Youth Mental Health (CYMH) Service Areas into five groupings, corresponding to the MCYS Integrated Service Regions (ISRs)
  • a method for using the TopoJSON files to visualize the CYMH Service Areas within the separate ISRs
  • the addition of a responsive framework to ensure that our visualizations accommodate to the display capabilities of various PCs, laptops, tablets, and smart phones

In this post, we describe how to enhance the functionality of our visualization of the Integrated Service Regions and CYMH Service Areas and Integrated across the entire province of Ontario.

Consolidated Geodata Files for CYMH Service Areas in Ontario

In a previous post, we provided a set of GeoJSON and TopoJSON files for the individual CYMH Service Areas and their groupings into the five distinct ISRs.

For present purposes, we have created two geodata files – in GeoJSON and TopoJSON formats – for the entire set of CYMH Service Areas in Ontario.

Field Property of the CYMH Service Area
area_name Name
id ID
isr Integrated Service Region
color Color

Simple Maps of the CYMH Service Areas

By way of a quick review, let’s start with a simple map outlining the CYMH Service Areas in Ontario:

<!DOCTYPE html>
<meta charset="utf-8">
<style>

/* CSS goes here */

/* define the container element for our map */
#map {
 margin:5% 5%;
 border:2px solid #000;
 border-radius: 5px;
 height:100%;
 overflow:hidden;
 background: #FFF;
}

</style>
<body>

/* create the container element #map */
<div id="map"></div>

/* load Javascript libraries for D3 and TopoJSON */
<script src="//d3js.org/d3.v3.min.js"></script>
<script src= "//d3js.org/topojson.v1.min.js"></script>

<script> // begin Javascript for visualizing the geo data

/* define some global variables */

var topo, projection, path, svg, g;

/* 1. Set the width and height (in pixels) based on the offsetWidth property of the container element #map */

var width = document.getElementById('map').offsetWidth;
var height = width / 2;

/* Call function setup() to create empty root SVG element with width, height of #map */

setup(width,height);

function setup(width,height){

/* 2. Create an empty root SVG element */

svg = d3.select('#map').append('svg')
 .style('height', height + 'px')
 .style('width', width + 'px')
 .append('g');

g = svg.append('g');

} // end setup()

/* 3. Define Unit Projection using Albers equal-area conic projection */

var projection = d3.geo.albers()
 .scale(1)
 .translate([0,0]);

/* 4. Define the path generator - to format the projected 2D geometry for SVG */

var path = d3.geo.path()
 .projection(projection);

/* 5.0 Start function d3.json() */
/* 5.1 Load the TopoJSON data file */

d3.json("http://cartoserve.com/maps/ontario/cymhsas33_data/cymhsas_topo.json", function(error, cymhsas_topo) {
if (error) return console.error(error);

/* 5.2 Convert the TopoJSON data back to GeoJSON format */
/* and render the map using Unit Projection */

var areas_var = topojson.feature(cymhsas_topo, cymhsas_topo.objects.cymhsas_geo);

/* 5.2.1 Calculate new values for scale and translate using bounding box of the service areas */
 
var b = path.bounds(areas_var);
var s = .95 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0][1]) / height);
var t = [(width - s * (b[1][0] + b[0][0])) / 2, (height - s * (b[1][1] + b[0][1])) / 2];

/* 5.2.2 New projection, using new values for scale and translate */
projection
 .scale(s)
 .translate(t);

/* redefine areas_var in terms of the .features array, assign array to topo */
var areas_var = topojson.feature(cymhsas_topo, cymhsas_topo.objects.cymhsas_geo).features;

topo = areas_var;

/* make the map by calling our draw() function initially within the d3.json callback function */
 
draw(topo);

}); // end function d3.json


function draw(topo) {

 var service_area = g.selectAll(".area_name").data(topo);

 service_area.enter().insert("path") 
 .attr("class", "service_area")
 .attr("d", path);

} // end function draw()

</script> // end Javascript for visualizing the geo data

Giving us the following visualization of 33 Service Areas [actual web page]:

CYMH Service Areas 33
Figure 1. Basic outline of 33 CYMH Service Areas in Ontario.

Note that the Javascript for creating our map includes three functions:

  • d3.json() – a built-in function to load geo data from a TopoJSON file
  • setup() – our function to create an empty root SVG element
  • draw() – our function to render the geo data

By adding just two lines of code to our draw() function, we can colour the CYMH Service Areas:


...
function draw(topo) {
 var service_area = g.selectAll(".area_name").data(topo);
 service_area.enter().insert("path") 
 .attr("class", "service_area")
 .attr("d", path)
 .attr("id", function(d,i) { return d.id; })
 .style("fill", function(d, i) { return d.properties.color; });

...

Giving us this figure [actual web page]:

CYMH Service Areas 33 colour
Figure 2. Thirty-three CYMH Service Areas in Ontario.

Adding Tooltips

Figures 1 and 2 suffer from one obvious shortcoming: none of the CYMH Service Areas is labelled. Unfortunately, our approach to labeling the CYMH Service Areas within a single Integrated Service Region breaks down when we have to contend with the scale of Ontario taken as a whole:

cymhsas02.html w labels screenshot
Figure 3. CYMH Service Areas cluttered with fixed labels.

Our best alternative is to use a tooltips to display the name of a CYMH Service Area when the user’s mouse hovers over the corresponding area of the visualization. Adding this functionality requires two sorts of modification of our Javascript.

First, we must style the Tooltips, including:

  • styling the <div> container element corresponding to the text box within which the name of the CYMH Service Area will be displayed
  • styling the text that is displayed when the user’s mouse is hovering over a service area
  • styling the tooltip so that it is hidden when the user’s mouse is not hovering over any service area

… like so:

<style>

...

/* style of the text box containing the tooltip */

div.tooltip {
 color: #222; 
 background: #fff; 
 padding: .5em; 
 text-shadow: #f5f5f5 0 1px 0;
 border-radius: 2px; 
 box-shadow: 0px 0px 2px 0px #a6a6a6; 
 opacity: 0.9; 
 position: absolute;
}

/* style of the text displayed in text box of the tooltip when mouse is hovering over a CYMH Service Area */

.service_area:hover{ stroke: #fff; stroke-width: 1.5px; }

.text{ font-size:10px; }

/* otherwise, the text box of the tooltip is hidden */
.hidden { 
 display: none; 
}

</style>

Second, we must modify our draw() function to display/hide tooltips in response to the user’s mousemove and mouseout behaviours:


<script>

var tooltip = d3.select("#map").append("div").attr("class", "tooltip hidden");

...

function draw(topo) {

var service_area = g.selectAll(".area_name").data(topo);

service_area.enter().insert("path")
.attr("class", "service_area")
.attr("d", path)
.attr("id", function(d,i) { return d.id; })
.style("fill", function(d, i) { return d.properties.color; })
.attr("title", function(d,i) { return d.properties.area_name; });

/* define offsets for displaying the tooltips */
var offsetL = document.getElementById('map').offsetLeft+20;
var offsetT = document.getElementById('map').offsetTop+10;

/* toggle display of tooltips in response to user mouse behaviours*/
service_area
//mousemove behaviour
.on("mousemove", function(d,i) {
var mouse = d3.mouse(svg.node()).map( function(d) { return parseInt(d); } );
tooltip.classed("hidden", false)
.attr("style", "left:"+(mouse[0]+offsetL)+"px;top:"+(mouse[1]+offsetT)+"px")
.html(d.properties.area_name);
}) // end mousemove
// mouseout behaviour
.on("mouseout", function(d,i) {
tooltip.classed("hidden", true);
}); // end mouseout

} // end draw()

Giving us this sort of visualization [interactive web page]:

CYMH Service Areas 33 tooltipsr
Figure 4. Thirty-three Colour CYMH Service Areas in Ontario with Tooltips.

Next time: We’ll enhance the functionality of our visualization to allow the user to pan and zoom our map of the CYMH Service Areas in Ontario.

Installing Bootstrap

You can install and manage Bootstrap’s Less, CSS, JavaScript, and fonts using npm:

$ npm install bootstrap

Note: require('bootstrap') will load all of Bootstrap’s jQuery plugins onto the jQuery object. You can manually load Bootstrap’s jQuery plugins individually by loading the /js/*.js files under the package’s top-level directory.

Compiling CSS and JavaScript

Bootstrap uses Grunt for its build system, with convenient methods for working with the framework. It’s how we compile our code, run tests, and more.

Installing Grunt

To install Grunt, you must first download and install node.js (which includes npm). npm stands for node packaged modules and is a way to manage development dependencies through node.js.

Then, from the command line:

  1. Install grunt-cli globally with npm install -g grunt-cli.
  2. Navigate to the root /root/node_modules/bootstrap/ directory, then run npm install. npm will look at the package.json file and automatically install the necessary local dependencies listed there.

When completed, you’ll be able to run the various Grunt commands provided from the command line.

grunt dist (Just compile CSS and JavaScript)

Regenerates the /dist/ directory with compiled and minified CSS and JavaScript files. As a Bootstrap user, this is normally the command you want.

Basic template

Start with this basic HTML template, or modify these examples.

Copy the HTML below to begin working with a minimal Bootstrap document.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
    <title>Bootstrap 101 Template</title>

    <!-- Bootstrap -->
    <link href="css/bootstrap.min.css" rel="stylesheet">

    <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
    <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
    <!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
    <![endif]-->
  </head>
  <body>
    <h1>Hello, world!</h1>

    <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
    <!-- Include all compiled plugins (below), or include individual files as needed -->
    <script src="js/bootstrap.min.js"></script>
  </body>
</html>

Examples

Build on the basic template above with Bootstrap’s many components. We encourage you to customize and adapt Bootstrap to suit your individual project’s needs. Get the source code for every example below by downloading the Bootstrap repository. Examples can be found in the docs/examples/ directory.

Visualizing the MCYS Service Areas within Integrated Service Regions Using D3.geo

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In a previous posting, we described our method for partitioning the Shapefile archive (cymh_shapefile.zip cymh_service_areas_after_march_9_2015.zip) of the thirty-four thirty-three MCYS Children and Youth Mental Health Service Areas (CYMHSAs) into five groupings, corresponding to the MCYS Integrated Service Regions (ISRs). We also provided the GeoJSON and TopoJSON files for the individual CYMH Service Areas and their groupings into Integrated Service Regions.

Now we’ll illustrate how to use the TopoJSON files and d3.geo to visualize the CYMH Service Areas within their respective Integrated Service Regions.

In the template below,  we’ve highlighted in red any values that relate to the Integrated Service Region of interest and we’ve highlighted in blue any values that relate to the CYMH Service Areas within the Integrated Service Region:

<!DOCTYPE html>
<meta charset="utf-8">
<style>

/* CSS goes here */

/* fill the Service Areas using colour scheme for their corresponding Integrated Service Region*/
.<ISR>_geo.<ServiceArea-1> { fill: <colour-1>; }
.<ISR>_geo.<ServiceArea-2> { fill: <colour-2>; }
...
.<ISR>_geo.<ServiceArea-n> { fill: <colour-n>; }

/* style the Service Area boundaries */
.sa_boundary {
  fill: none;
  stroke: #000;
  stroke-width: 1.5px;
  stroke-linejoin: round;
}

/* style the Service Area labels */

.area-label {
 fill: #000;
 fill-opacity: .9;
 font-size: 10px;
 text-anchor: middle;
}

</style>
<body>
<script src="//d3js.org/d3.v3.min.js" charset="utf-8"></script>
<script src= "//d3js.org/topojson.v1.min.js"></script>

<script>

/* 1. Set the width and height (in pixels) of the canvas */
var width = 960,
    height = 500;
 
/* 2. Create an empty root SVG element */

var svg = d3.select("body").append("svg") .attr("width", width) .attr("height", height);

/* 3. Define the Unit projection to project 3D spherical coordinates onto the 2D Cartesian plane.Note - we use the Albers equal-area conic projection. */
var projection = d3.geo.albers()
    .scale(1)
    .translate([0, 0]);

/* 4. Define the path generator - to format the projected 2D geometry for SVG */
var path = d3.geo.path()
    .projection(projection);

/* 5.0 Open the d3.json callback, and
/* 5.1 Load the TopoJSON data file. */

d3.json("<ISR>_topo.json", function(error, <ISR>_topo) {
if (error) return console.error(error);

/* 5.2 Convert the TopoJSON data back to GeoJSON format */

  var areas_var = topojson.feature(<ISR>_topo, <ISR>_topo.objects.<ISR>_geo);
/* 5.2.1 Calculate new values for scale and translate using bounding box of the service areas */
 
var b = path.bounds(areas_var);
var s = .95 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0][1]) / height);
var t = [(width - s * (b[1][0] + b[0][0])) / 2, (height - s * (b[1][1] + b[0][1])) / 2];

/* 5.2.2 New projection, using new values for scale and translate */
projection
   .scale(s)
   .translate(t);

/* 5.3 Bind the GeoJSON data to the path element and use selection.attr to set the "d" attribute to the path data */

svg.append("path")
.datum(areas_var)
.attr("d", path);

/* 6. Draw the boundaries of the Service Areas */
  svg.append("path")
      .datum(topojson.mesh(<ISR>_topo, <ISR>_topo.objects.<ISR>_geo, function(a, b) { return a !== b; }))
      .attr("class", "sa_boundary")
      .attr("d", path);

/* 7 Colour the Service Areas */

  svg.selectAll(".<ISR>_geo")
      .data(topojson.feature(<ISR>_topo, <ISR>_topo.objects.<ISR>_geo).features)
      .enter().append("path")
      .attr("class", function(d) { return "<ISR>_geo " + d.id; })
      .attr("d", path);

/* 8 Label the Service Areas */

 svg.selectAll(".area-label")
 .data(topojson.feature(<ISR>_topo, <ISR>_topo.objects.<ISR>_geo).features)
 .enter().append("text")
 .attr("class", function(d) { return "area-label " + d.id; })
 .attr("transform", function(d) { return "translate(" + path.centroid(d) + ")"; })
 .attr("dy", ".35em")
 .text(function(d) { return d.properties.area_name; });

/* 9. Close the d3.json callback */

});

</script>

The following table presents the five MCYS Integrated Service Regions and their respective CYMH Service Areas. Drawing upon the resources of ColorBrewer, we’ve assigned a different hue to every Integrated Service Region and a distinctive colour of that hue to every member CYMH Service Area:

Central Region
Service Area Colour
Dufferin/Wellington #fcbba1
Halton #fc9272
Peel #fb6a4a
Simcoe #ef3b2c
Waterloo #fee0d2
York #a50f15
East Region
Service Area Colour
Durham #efedf5
Frontenac/Lennox and Addington #9e9ac8
Haliburton/Kawartha Lakes/Peterborough #dadaeb
Hastings/Prince Edward/Northumberland #bcbddc
Lanark/Leeds and Grenville #807dba
Ottawa #6a51a3
Prescott and Russell #54278f
Renfrew #54278f
Stormont, Dundas and Glengarry #3f007d
North Region
Service Area Colour
Algoma #74c476
Greater Sudbury/Manitoulin/Sudbury #a1d99b
James Bay Coast #238b45
Kenora/Rainy River #00441b
Nipissing/Parry Sound/Muskoka #c7e9c0
Thunder Bay #006d2c
Timiskaming/Cochrane

Cochrane/Timiskaming (including James Bay Coast)

#41ab5d
Toronto Region
Service Area Colour
Toronto #f16913
West Region
Service Area Colour
Brant #08519c
Chatham-Kent #c6dbef
Elgin/Oxford #4292c6
Essex #deebf7
Grey/Bruce #08519c
Haldimand-Norfolk #2171b5
Hamilton #08306b
Huron/Perth #2171b5
Lambton #9ecae1
Middlesex #6baed6
Niagara #4292c6

So, now let’s display the Integrated Service Regions and their member CYMH Service Areas:

Central Region (actual rendering):

MCYS Central Region and Member CYMH Service Areas
Figure 1. MCYS Central Region and Member CYMH Service Areas.

East Region (actual rendering):

MCYS Central Region and Member CYMH Service Areas
Figure 2. MCYS Central Region and Member CYMH Service Areas.

Toronto Region (actual rendering):

MCYS Toronto Region
Figure 3. MCYS Toronto Region.

North Region – 7 Service Areas (actual rendering):

MCYS North Region and Member CYMH Service Areas
Figure 4a. MCYS North Region and 7 Member CYMH Service Areas.

North Region – 6 Service Areas (actual rendering):

CYMH Service Areas in North Region - 33 - screenshot
Figure 4b. MCYS North Region and 6 Member CYMH Service Areas.

West Region (actual rendering):

MCYS West Region and Member CYMH Service Areas
Figure 5. MCYS West Region and Member CYMH Service Areas.

Next time: We’ll add some functionality so that users can interact with our maps.

MCYS Service Areas and Integrated Service Regions in GeoJSON and TopoJSON Formats

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

Here we provide geospatial data for the MCYS Integrated Service Regions and their constituent Children and Youth Mental Health Service Areas in GeoJSON and TopoJSON file formats:

Central Region GeoJSON TopoJSON
Dufferin/Wellington GeoJSON TopoJSON
Halton GeoJSON TopoJSON
Peel GeoJSON TopoJSON
Simcoe GeoJSON TopoJSON
Waterloo GeoJSON TopoJSON
York GeoJSON TopoJSON
East Region GeoJSON TopoJSON
Durham GeoJSON TopoJSON
Frontenac/Lennox and Addington GeoJSON TopoJSON
Haliburton/Kawartha Lakes/Peterborough GeoJSON TopoJSON
Hastings/Prince Edward/Northumberland GeoJSON TopoJSON
Lanark/Leeds and Grenville GeoJSON TopoJSON
Ottawa GeoJSON TopoJSON
Prescott and Russell GeoJSON TopoJSON
Renfrew GeoJSON TopoJSON
Stormont, Dundas and Glengarry GeoJSON TopoJSON
North Region – 7 Service Areas

North Region – 6 Service Areas

GeoJSON

GeoJSON

TopoJSON

TopoJSON

Algoma GeoJSON TopoJSON
Greater Sudbury/Manitoulin/Sudbury GeoJSON TopoJSON
James Bay Coast GeoJSON TopoJSON
Kenora/Rainy River GeoJSON TopoJSON
Nipissing/Parry Sound/Muskoka GeoJSON TopoJSON
Thunder Bay GeoJSON TopoJSON
Timiskaming/Cochrane

Cochrane/Timiskaming (including James Bay Coast)

GeoJSON

GeoJSON

TopoJSON

TopoJSON

Toronto Region GeoJSON TopoJSON
West Region GeoJSON TopoJSON
Brant GeoJSON TopoJSON
Bruce/Grey GeoJSON TopoJSON
Chatham-Kent GeoJSON TopoJSON
Elgin/Oxford GeoJSON TopoJSON
Essex GeoJSON TopoJSON
Haldimand-Norfolk GeoJSON TopoJSON
Hamilton GeoJSON TopoJSON
Huron/Perth GeoJSON TopoJSON
Lambton GeoJSON TopoJSON
Middlesex GeoJSON TopoJSON
Niagara GeoJSON TopoJSON

Geospatial features of the MCYS Children and Youth Mental Health Service Areas

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In 2014-15, the Ministry of Children and Youth Services (MCYS) defined two administrative views of mental health services for children and youth in Ontario:

The provincial government has published geospatial data for the MCYS’s Integrated Service Regions (ISRs) and Children and Youth Mental Health Service Areas (CYMHSAs) in Shapefile format.

Here, we’ll work only with the geospatial data for the CYMHSAs, publishing three sorts of files for use in d3.geo:

  1. Converting the Shapefile archive (cymh_shapefile.zip cymh_service_areas_after_march_9_2015.zip) for the entire set of thirty-four MCYS Children and Youth Mental Health Service Areas into a single TopoJSON file (cymhsas_topo.json)
  2. Partitioning the Shapefile archive in #1 into five groups of Service Areas, corresponding to the Integrated Service Regions, and converting these five groups into TopoJSON files:
    1. central_topo.json
    2. east_topo.json
    3. north_topo.json
    4. toronto_topo.json
    5. west_topo.json
  3. Partitioning the Shapefile archive in #1 into thirty-four thirty-three groups, corresponding to the individual MCYS Children and Youth Mental Health Service Areas, and converting these groups into TopoJSON files

Shapefile archive for 34 33 CYMHSAs

The Shapefile archive  for the thirty-four thirty-three MCYS CYMHSAs (cymh_service_areas_after_march_9_2015.zip) contains four files:

  • CYMH Service Areas.shp — shape format
  • CYMH Service Areas.shx — shape index format
  • CYMH Service Areas.dbf — attribute format
  • CYMH Service Areas.prj — projection format: the coordinate system and projection information, expressed in well-known text format

we rename the four CYMH Service Areas.* files cymh-service-areas.*

Converting the Shapefile archive to GeoJSON and TopoJSON format

To use d3.geo to visualize the CYMHSAs, we first use ogr2ogr to convert the Shapefiles to GeoJSON format, and then use topojson to convert the GeoJSON file to TopoJSON format:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON cymhsas_geo.json cymh-service-areas.shp

topojson -o cymhsas_topo.json --id-property area_name --properties -- cymhsas_geo.json

Notes:

  • After running ogr2ogr, we use a text editor to replace the feature property Name with area_name in the file cymhsas_geo.json 1
  • The --id-property switch in topojson is used to promote the feature property area_name to geometry id in the file cymhsas_topo.json
  • We use a text editor to remove any special characters (e.g. spaces, “,”, “/”, “-“) from the value of the geometry id, e.g. “"id": "Haliburton/Kawartha Lakes/Peterborough"" becomes “"id": "HaliburtonKawarthaLakesPeterborough"” in cymhsas_topo.json

Partitioning the Shapefile archive of CYMHSAs into ISRs

We illustrate the partitioning of the Shapefile archive into groups of Children and Youth Mental Health Service Areas corresponding to the five Integrated Service Regions with the script for the North Region.
For 34 CYMHM Service Areas:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON -where "Name =  'Algoma' OR Name = 'Greater Sudbury/Manitoulin/Sudbury' OR Name = 'James Bay Coast' OR Name = 'Kenora/Rainy River'  OR Name = 'Nipissing/Parry Sound/Muskoka' OR Name = 'Thunder Bay' OR Name =  'Timiskaming/Cochrane'" north_geo.json cymh-Service-Areas

topojson -o north_topo.json --id-property area_name --properties -- north_geo.json

For 33 CYMH Service Areas:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON -where "ServiceA00 =  'Algoma' OR Name = 'Greater Sudbury/Manitoulin/Sudbury' OR Name = 'Kenora/Rainy River'  OR Name = 'Nipissing/Parry Sound/Muskoka' OR Name = 'Thunder Bay' OR Name =  'Cochrane/Timiskaming'" north_geo.json cymh-service-areas.shp

topojson -o north_topo.json --id-property area_name --properties -- north_geo.json

Note:

  • The MCYS uses various spelling conventions for compound CYMHSAs; the authoritative list of CYMHSAs naming conventions is found in CYMH-Service-Areas.dbf in the Shapefile archive.

Partitioning the Shapefile archive of CYMHSAs into individual CYMHSAs

Partitioning of the Shapefile archive into individual Children and Youth Mental Health Service Areas is straightforward, e.g.:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON -where "Name = 'Toronto'" toronto_geo.json cymh-service-areas.shp

topojson -o toronto_topo.json --id-property area_name --properties -- toronto_geo.json

I’ve made a complete set of these GeoJSON and TopoJSON available for anyone to use freely.

 

 

Geospatial Features of the MCYS Integrated Service Regions

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

In 2014-15, the Ministry of Children and Youth Services (MCYS) defined two administrative views of mental health services for children and youth in Ontario:

The provincial government has published geospatial data for the MCYS’s Integrated Service Regions (ISRs) and Children and Youth Mental Health Service Areas (CYMHSAs) in Shapefile format. Here, we’ll work only with the geospatial data for the ISRs.

Shapefile format

The Shapefile archive for the ISRs (mcys_integrated_regions.zip) contains four files:

  • mcys_integrated_regions.shp — shape format
  • mcys_integrated_regions.shx — shape index format
  • mcys_integrated_regions.dbf — attribute format
  • mcys_integrated_regions.prj — projection format: the coordinate system and projection information, expressed in well-known text format

For ease of reference, we rename the four mcys_integrated_regions.* files isrs.*.

Converting Shapefiles to GeoJSON format

To use d3.geo to visualize the ISRs, we first convert the Shapefiles to GeoJSON format files, using ogr2ogr. There are three steps:

  1. Determine the Spatial Reference System (SRS) used by the Shapefile
  2. Set the -t_srs switch in ogr2ogr to output the GeoJSON file using this SRS
  3. Rename variables for ease of use

The projection format file for the ISRs specifies:

GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]]

We use Prj2EPSG, a simple online service to convert the projection information contained in the .prj file into standard EPSG codes for the corresponding spatial reference system. From this we determine that the specification contained in isrs.prj corresponds to EPSG 4269 – GCS_North_American_1983.

We now use the -t_srs switch in ogr2ogr to transform the output using the spatial reference system specified in isrs.prj:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON isrs_geo.json isrs.shp

The GeoJSON format file is isrs_geo.json.

Finally, we use a text editor to give more meaningful names to a few variables in the GeoJSON files and to express them in all lowercase letters:

Original variable Modified variable
Region region_name
Order order

Converting GeoJSON files to TopoJSON format

We use topojson to convert the GeoJSON file isrs_geo.json to a TopoJSON format file:

topojson -o isrs_topo.json --id-property region_name --properties -- isrs_geo.json

The --id-property switch in topojson is used to promote the feature property "region_name" to geometry id status in the TopoJSON file isrs_topo.json.

We may now visualize the MCYS Integrated Service Regions by applying d3.geo to isrs_topo.json.

Visualizing the MCYS Integrated Service Regions Using d3.geo

Preamble

This article was originally posted on May 14, 2016 and revised on July 28, 2016 to take account of changes in the geospatial representation of Children and Youth Mental Health Service Areas in Ontario. For more details, see … and then there were 33.

Introduction

In the past few years, a wide variety of free and open source software (FOSS) tools for visualizing geospatial data have become available.  For many people like me who work in human services, coming to know how to use these tools even in rudimentary ways represents a steep learning-curve. Here, I illustrate the use of one of these tools, d3.geo, to visualize a geospatial view of children and youth services in Ontario. 1

In 2014-15 the Ministry of Children and Youth Services (MCYS) defined two administrative views of Ontario:

The five Integrated Service Regions (ISRs) combined nine previous Service Delivery Division regions and four Youth Justice Services Division regions. The ISRs are integrated with the five regional boundaries of the Ministry of Community and Social Services (MCSS).

The thirty-four thirty-three Children and Youth Mental Health Service Areas (CYMHSAs) were defined after a thorough review, including an assessment of Statistics Canada’s census divisions and projected population and children and youth.

The Ontario government has published two Shapefile archives – mcys_integrated_regions.zip and cymh_shapefile.zip and cymh_service_areas_after_march_9_2015.zip – that define the geospatial boundaries of the ISRs and the CYMHSAs, respectively. For now, we’re going to work only with the Shapefile archive for the ISRs.

Before we can visualize the ISRs using d3.geo, we need to convert the Shapefile format archive – mcys_integrated_regions.zip - to a TopoJSON format file – isrs_topo.json. (For more details of converting geospatial data from one file format to another, see Geospatial Features of the MCYS Integrated Service Regions).

A Simple Map

So, let’s use d3.geo to visualize the MCYS Integrated Service Regions in Ontario.

HTML template

In the same directory as the isrs_topo.json file, we create a file – isrs01.html – using the following template:


<!DOCTYPE html>
<meta charset="utf-8">
<style>

/* CSS goes here */

</style>
<body>
<script src="//d3js.org/d3.v3.min.js" charset="utf-8"></script>
<script src= "//d3js.org/topojson.v1.min.js"></script>

/*                                                           */
<script>
/* JavaScript for reading and rendering data goes here */
</script>

d3 supports the two main standards for rendering two-dimensional geometry in a browser: SVG and Canvas. We prefer SVG because you can style SVG using CSS, and declarative styling is easier.

There are several steps involved in reading and rendering our data in SVG:

  1. Define the width and height (in pixels) of the canvas
  2. Create an (empty) root SVG element
  3. Define the projection, beginning with a “unit projection” with .scale(1) and .translate([0,0])
  4. Define the path generator
  5. Open the d3.json callback
    1. Load the TopoJSON data file
    2. Convert the TopoJSON data back to GeoJSON format
    3. Calculate new values for .scale() and .translate() to resize and centre the projection
    4. Bind the GeoJSON data to the path element and use selection.attr to set the “d” attribute to the path data
  6. Maybe do some other stuff
  7. Close the d3.json callback

If we modify our template – isrs01.html – by adding the Javascript to load and render isrs_topo.json in SVG, we obtain:


<!DOCTYPE html>
<meta charset="utf-8">
<style>

/* CSS goes here */

</style>
<body>
<script src="//d3js.org/d3.v3.min.js" charset="utf-8"></script>
<script src= "//d3js.org/topojson.v1.min.js"></script>

<script>

/* 1. Set the width and height (in pixels) of the canvas */
var width = 960,
    height = 1160;
 
/* 2. Create an empty root SVG element */

var svg = d3.select("body").append("svg") .attr("width", width) .attr("height", height);

/* 3. Define the Unit projection to project 3D spherical coordinates onto the 2D Cartesian plane - HERE we use the Albers equal-area conic projection. */
var projection = d3.geo.albers()
    .scale(1)
    .translate([0, 0]);

/* 4. Define the path generator - to format the projected 2D geometry for SVG */
var path = d3.geo.path()
    .projection(projection);

/* 5.0 Open the d3.json callback, and
/* 5.1 Load the TopoJSON data file. */

d3.json("isrs_topo.json", function(error, isrs_topo) {
if (error) return console.error(error);

/* 5.2 Convert the TopoJSON data back to GeoJSON format */

  var regions_var = topojson.feature(isrs_topo, isrs_topo.objects.isrs_geo);
/* 5.2.1 Calculate new values for scale and translate using bounding box of the service areas */
 
var b = path.bounds(regions_var);
var s = .95 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0][1]) / height);
var t = [(width - s * (b[1][0] + b[0][0])) / 2, (height - s * (b[1][1] + b[0][1])) / 2];

/* 5.2.2 New projection, using new values for scale and translate */
projection
   .scale(s)
   .translate(t);

/* 5.3 Bind the GeoJSON data to the path element and use selection.attr to set the "d" attribute to the path data */

svg.append("path")
.datum(regions_var)
.attr("d", path);

/* 6 - 8 Some other stuff TBD later */

/* 9. Close the d3.json callback */

});

</script>

… which gives us this sort of basic rendering of the MCYS Integrated Service Regions [actual rendering]:

Figure 1. Basic rendering of the MCYS Integrated Service Regions.
Figure 1. Basic rendering of the MCYS Integrated Service Regions.

There are a few obvious improvements we can make. First, let’s draw the boundaries of the MCYS ISRs:


/* CSS goes here */

/* Define the boundary of an ISR as a 1.5 px wide white line, with a round line-join */
.isr_boundary {
  fill: none;
  stroke: #fff;
  stroke-width: 1.5px;
  stroke-linejoin: round;
}

...

/* Javascript for reading and rendering data in SVG */

...

/* 6. Draw the boundaries of the ISRs */
  svg.append("path")
      .datum(topojson.mesh(isrs_topo, isrs_topo.objects.isrs_geo, function(a, b) { return a !== b; }))
      .attr("class", "isr_boundary")
      .attr("d", path);

… giving us this sort of map [actual rendering]:

MCYS Integrated Service Regions with boundaries
Figure 2. Rendering of the MCYS Integrated Service Regions with boundaries.

… and then let’s label and colour the MCYS ISRs using the colour-scheme adopted by the MCYS:


/* CSS goes here */

/* fill the ISRs using the MCYS colour scheme */
.isrs_geo.Toronto { fill: #bd3f23; }
.isrs_geo.Central { fill: #fcb241; }
.isrs_geo.East { fill: #a083a7; }
.isrs_geo.West { fill: #e3839e; }
.isrs_geo.North { fill: #8dc73d; }

/* switch the colour of the ISR boundaries to black */
.isr_boundary {
  fill: none;
  stroke: #000;
  stroke-width: 1.5px;
  stroke-linejoin: round;
}

/* style the Region label */

.region-label {
 fill: #000;
 fill-opacity: .9;
 font-size: 12px;
 text-anchor: middle;
}

...

/* Javascript for reading and rendering data in SVG */

...

/* 7 Colour the ISRs */

  svg.selectAll(".isrs_geo")
      .data(topojson.feature(isrs_topo, isrs_topo.objects.isrs_geo).features)
      .enter().append("path")
      .attr("class", function(d) { return "isrs_geo " + d.id; })
      .attr("d", path);

/* 8 Label the Integrated Service Regions */

 svg.selectAll(".region-label")
 .data(topojson.feature(isrs_topo, isrs_topo.objects.isrs_geo).features)
 .enter().append("text")
 .attr("class", function(d) { return "region-label " + d.id; })
 .attr("transform", function(d) { return "translate(" + path.centroid(d) + ")"; })
 .attr("dy", ".35em")
 .text(function(d) { return d.properties.region_name; });

giving us this sort of map [actual rendering]:

Figure 3. Labelling and rendering of the MCYS Integrated Service Regions in colour.
Figure 3. Labeling and rendering of the MCYS Integrated Service Regions in colour.

Wrap-Up

In this post, we use d3.geo – a free and open source software tool – to visualize a publicly-available geospatial dataset related to children and youth services in five regions across Ontario. Next time, we will use another dataset to visualize children and youth services at the level of thirty-four service areas in the province.

  1. Accessing d3.geo is easy – you simply include a single call to d3 in the body of an .html web page. Installing and using some of the tools for converting geospatial data from one format to another (in order to use d3.geo) is more complicated. For some more technical details, see  Installing tools for d3.geo – 20160306.

A problem using EPSG:26917 to convert *.shp files to *_geo.json format

Converting Shapefiles to GeoJSON format

To use d3.geo to visualize the ISRs and the CYMHSAs, we first convert their Shapefiles into GeoJSON format files, using ogr2ogr. There are three steps:

  1. Determine the Spatial Reference System (SRS) used by the Shapefile
  2. Set the -t_srs switch in ogr2ogr to output the GeoJSON file using this SRS
  3. Rename variables for ease of use

The projection format files for the ISRs and the CYMHSAs – isrs.prj and cymhsas.prj, respectively – specify:

GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]]

and

PROJCS["NAD_1983_UTM_Zone_17N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000],PARAMETER["False_Northing",0],PARAMETER["Central_Meridian",-81],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0],UNIT["Meter",1]]

We use Prj2EPSG, a simple online service to convert the projection information contained in these .prj files into standard EPSG codes for the corresponding spatial reference system(s). From this we determine that the specification contained in isrs.prj corresponds to EPSG 4269 – GCS_North_American_1983; likewise, cymhsas.prj corresponds to EPSG 26917 – NAD_1983_UTM_Zone_17N.

We now use the -t_srs switch in ogr2ogr to transform the output using the spatial reference system specified in isrs.prj and cymhsas.prj:

ogr2ogr -t_srs EPSG:4269 -f GeoJSON isrs_geo.json isrs.shp
ogr2ogr -t_srs EPSG:26917 -f GeoJSON cymhsas_geo.json cymhsas.shp

Unfortunately, “-t_srs EPSG:26917” seems to breaks ogr2ogr – and the cymhsas_geo.json file is corrupt. Consequently, we use EPSG 4269 for transforming cymhsas.shp. The issue with EPSG 26917 needs further research.

A related question: How does one merge two *_geo.json files if a different EPSG is used to convert their respective *.shp files?

Settling Smith, LJ – Captain Rock: Transplanting the Irish Agrarian Rebellion in Upper Canada, 1823-4

Settling Smith, LJ – Captain Rock: Transplanting the Irish Agrarian Rebellion in Upper Canada, 1823-4

Irish White Boys, from George Lillie Craik and C. MacFarlane, The Pictorial History of England: Being a History of the People as well as a History of the Kingdom, Vol. 5 (London: Charles Knight, 1849), 79. Public Domain via Google Books.

Settling Captain Rock: Transplanting the Irish Agrarian Rebellion in Upper Canada, 1823-4
Laura J. Smith

In the summer of 1824 the British Colonial Office instructed the Upper Canadian government to give a soon-to-arrive Irish emigrant named John Dundon a “gratuitous” land grant of 200 acres and provisions for a year.[1] Such assistance was not unusual. Assisted emigration programs targeting disbanded soldiers, dispossessed peasants, and unemployed craftsmen had already contributed substantially to the Upper Canadian settler population. But why did this particular emigrant warrant individual attention? A second letter sent a few weeks later went further: “John Dundon rendered some important services to the Irish government as an informer and… has been provided for as a settler.” Dundon was to be sent to the Lake Simcoe region, the letter instructed, for “the vicinity of the other settlers would be hardly safe for him.”[2]

John Dundon was a notorious “Captain Rock”; a leader within the Rockites, a secret agrarian protest group active in the Blackwater region of North County Cork in the mid-1820s. [3] Like similar Irish groups of the period, the Rockites agitated for social and economic change and particularly for the reform of access to and ownership of land. They targeted, usually through vigilante-style violence, all those who blocked the peasantry’s access to land. Dundon’s arrest and confession, which identified 50 local Rockites, and locations of substantial caches of arms, was a coup for beleaguered magistrates tasked with subduing the increasingly violent Rockites. Dundon’s information proved valuable, and consequently officials lobbied for a provision that might double as protection for Dundon and his family who it was noted, could no longer remain safely in Ireland.[4]

Smith: Emigrants Arrival at Cork
Emigrants Arrival at Cork–A Scene on the Quay. Illustrated London News, May 10, 1851. Public Domain via Steve Taylor at https://viewsofthefamine.wordpress.com
That Dundon was directed to Upper Canada is not surprising. The previous summer the British government had conveyed nearly 600 mostly Irish Catholics, the “other settlers” from the Blackwater to eastern Upper Canada under similar, but slightly less generous assistance terms.[5] Dubbed the “Peter Robinson emigrants” after the Upper Canadian who superintended their migration, they were the first of two groups of Irish Catholics sent to Upper Canada from the region.[6] Their assistance program was premised on contemporary ideas about the removal of “surplus” populations to ameliorate agrarian discontent and free land for modernization. The Blackwater region was targeted specifically because of the escalation of Rockite violence, and the intense lobbying of local landlords. Land use was changing, landholders argued, and the peasant farmer was obsolete. Such men, who in Ireland had rented a few acres for basic subsistence, were better settled in Canada where they could “cultivate the waste lands…and be useful members of society.” Left in Ireland, they were likely to turn into “bad subjects” who devoted their “time to Captain Rock and his associates.”[7]

There is no explicit evidence that any of the 1823 emigrants were Rockites, or that the assisted emigration scheme that year was used to transport troublemakers. Irish sources are inconclusive, and it appears more probable that participants, particularly a group of Irish Palatines, emigrated out of fear rather than complicity with the Rockites.[8] Though he insisted he chose only “paupers” who had agricultural experience, and thus had a chance to do well in Upper Canada, Peter Robinson was vague on the extent to which his emigrants could be implicated in the violence. Arguing that even the most “fiery” Irish male no matter his “former conduct,” could be tamed with a fresh start in Upper Canada, Robinson admitted that he had allowed local magistrates to select from the list of willing migrants those, “they [were] most desirous to get rid of.”[9] Such sentiments reflect an elitist view of the disposition toward anti-social behaviour that was both natural to the Irish character and nurtured by the Irish environment. For imperial officials explicit participation in secret societies was largely irrelevant for the state of Irish rural society made every inherently disorderly Irish peasant a potential Rockite.

The assisted emigrants were met with similar sentiments in Upper Canada, where perceptions and stereotypes that Irish Catholics were violent or easily provoked into violence persisted. Certainly their new neighbours needed little proof that the 1823 emigrants were Rockites or carried politically problematic baggage. The assistance the Irish had received was seen locally as underserved, Peter Robinson reported; the Irish, it was said, “had done nothing to entitle themselves to any bounty from the government, further than keeping their own country disturbed.”[10]

Smith Priest’s blessing nypl.digitalcollections.510d47e1-37f2-a3d9-e040-e00a18064a99.001.w
Irish Emigrants Leaving Home–The Priest’s Blessing (1851) New York Public Library Digital Collections.
When a few members of the 1823 group were implicated in a tavern riot in the spring of 1824, locals saw confirmation that the violence of rural Ireland had been transplanted in their midst. Area officials immediately solicited military intervention certain that an insurrection was afoot. Drawing on tropes familiar to anyone who had read accounts of rural agitation in Ireland, the Montreal Herald depicted the emigrants as mindlessly violent, defiant of all authority, and opposed to the natural order of settlement life. Their violence was indicative, the paper implied, of those “permanent political feelings incident to a great proportion of Irish Emigrants.”[11] Despite absolution from a colonial investigator following the tavern riot, and clear evidence of settlement success, his fellow Blackwater emigrants continued to be plagued by the stigma of violence and questionable politics decades after their initial arrival.[12]

It is little wonder then that John Dundon was to be directed away from eastern Upper Canada and the emigrants who were under such close scrutiny and stigma. Engaged in a potentially large-scale and long-term project of assisted migration and settlement of Irish Catholics to Upper Canada, the Colonial Office wished to avoid further scrutiny of the goals of its colonial emigration policy, particularly accusations that it was prioritizing the displacement of Irish violence over the safety of Upper Canada. The settlement of John Dundon, an actual confessed agrarian rebel, had to be done carefully and quietly. It appears they were successful. It is not clear if Dundon claimed his reward in Upper Canada.[13] Land records for the colony offer a few misspelled, but inconclusive possibilities in Tiny, Vespra, and Mariposa Townships.[14] Of course it is possible he changed his name, or continued to the United States, but regardless John Dundon, it would seem, slipped into obscurity.

Considered together, the stories of John Dundon and the 1823 emigrants reveal the extent to which scrutiny and stigma of recent migrants, particularly those from disturbed regions and for whom state assistance is given, is nothing new. The state-sponsored migration and settlement of the 1823 emigrants sparked discussions about security, funding, reception and cultural integration, issues that remain important in 2016.

Laura J. Smith is a PhD Candidate in the department of history at the University of Toronto. Her forthcoming dissertation is entitled: “Unsettled Settlers: Irish Catholics and Irish Catholicism on the British North American settlement frontier, 1820-1840.” She can be found on Twitter @l4smith.

[1] Much of this post is drawn from the forthcoming: Laura J. Smith, “The Ballygiblins: British emigration policy, Irish violence, and immigrant reception in Upper Canada,” Ontario History, Vol. CVIII No. 1, Spring 2016.

[2] Archives of Ontario, Upper Canada Sundries, C-4613, p. 35320, Peter Robinson to Major G. Hillier, 12 June 1824; p. 35793, Peter Robinson to Major G. Hillier, 1 August 1824; Archives of Ontario, Peter Robinson Fonds, MS12, Reel 1, Major G. Hillier to Peter Robinson, 24 October 1824; Peter Robinson to Robert Wilmot Horton, 7 December 1824.

[3] James S. Donnelly, Captain Rock: The Irish Agrarian Rebellion of 1821-1824 (Madison: University of Wisconsin Press, 2009) is the definitive history of the Rockite rebellion.

[4] National Archives of Ireland, State of the Country Papers, 2514/24, Finch to Arbuthnot, 19 July 1823.

[5] Male assisted emigrants over 18 were given 70 acres of land in the Bathurst District. A provision for the an additional 30 acres was to be made once the emigrants proved themselves to be “industrious and prudent.”

[6] The second assisted emigration in 1825 was larger, and sent approximately 2000 emigrants to the area around what is now known as Peterborough.

[7] Archives of Ontario, Peter Robinson fonds, MS 12, Reel 1, Kingston to Robinson, 19 December 1824.

[8] A search of the State of the Country Papers at the National Archives of Ireland revealed no explicit participation in Rockite violence on the part of the selected 1823 emigrants. For more on Irish Palatines see, Carolyn A. Heald, The Irish Palatines in Ontario: Religion, Ethnicity, and Rural Migration, (Gananoque: Langdale Press, 1994).

[9] Library and Archives Canada, Colonial Office 384/12 f. 21, Peter Robinson to Robert Wilmot Horton, 9 June 1823; Archives of Ontario, Peter Robinson fonds, MS 12, Reel 1, undated report by Peter Robinson to Lord Bathurst.

[10] Reports from the Select Committee on Emigration, (Great Britain: Parliament, House of Commons, Select Committee on Emigration from United Kingdom, 1826) p. 332.

[11] Montreal Herald, 5, 12, and 15 May 1824.

[12] Thirteen years later, when news of the aborted Upper Canadian rebellion reached London, observers there immediately assumed, entirely erroneously, that the 1823 emigrants would be implicated in the affair. See Robert Wilmot Horton, Ireland and Canada, supported by Local evidence, (London, 1839) p. 76-78, https://archive.org/details/cihm_21752. Perhaps anticipating questions about their loyalty, the Irish Catholics in the region sent a loyalty address to the Lieutenant Governor. See, Colin Read and Ronald J. Stagg ed., The Rebellion in Upper Canada: A Collection of Documents (Ottawa: Champlain Society, 1988) 263-265.

[13] Archives of Ontario, Peter Robinson fonds, MS 12, Reel 1, Major G. Hillier to Peter Robinson, 24 October 1824 indicated that Dundon had yet to arrive but was expected imminently.

[14] Archives of Ontario, Ontario Land Records Index, 1789-1920.

Featured Image: Irish White Boys, from George Lillie Craik and C. MacFarlane, The Pictorial History of England: Being a History of the People as well as a History of the Kingdom, Vol. 5 (London: Charles Knight, 1849), 79. Public Domain via Google Books.

About these ads

Share this:
3Share on Facebook (Opens in new window)3Click to share on Twitter (Opens in new window)Click to email (Opens in new window)Click to share on Reddit (Opens in new window)Click to share on Google+ (Opens in new window)Click to share on Pocket (Opens in new window)Click to share on Pinterest (Opens in new window)
File M and the Straightness of the Settler State in Early Canada
File M and the Straightness of the Settler State in Early Canada
In “Research”
The “Canadian Revolution,” the Early American Republic, and … Slavery?
The “Canadian Revolution,” the Early American Republic, and … Slavery?
In “Historiography”
The Quebec Invasion as Religious Encounter
The Quebec Invasion as Religious Encounter
In “Research”
on April 4, 2016 x
Post Tags:
19th Century 16
British Empire 1
British North America 15
Canadian History 28
Catholicism 2
Early Canadian History 33
History 24
Immigration 2
Irish History 1
Laura J. Smith 1
North America 21
Political History 5
transnational history 9
Upper Canada 5
Continue the conversation …

Enter your comment here…

Search …
s
Recent Posts

Settling Captain Rock: Transplanting the Irish Agrarian Rebellion in Upper Canada, 1823-4
A Northern Chorus: The Canadian Turn in Early American History
Dartmouth College and Canada: The Problem of National Historiographies
Let’s work together: A loyalist historian from Canada responds to American scholars
Canadian Fugitive Slave Advertisements: An Untapped Archive of Resistance
Archives

April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
May 2015
Categories

Book Previews
Conference Recaps
Digital History
Historiography
Primary Sources
Research
Reviews
Teaching
Uncategorized
Unrest Violence Social Order
Meta

Register
Log in
Entries RSS
Comments RSS
Create a free website or blog at WordPress.com.
x
The Clear News Theme.

Create a free website or blog at WordPress.com.
Follow
Follow “Borealia”

Get every new post delivered to your Inbox.

Join 1,461 other followers
Enter your email address

Sign me up

Build a website with WordPress.com

Watts, Duncan – The organizational spectroscope – Medium 20160401

Watts, Duncan – The organizational spectroscope – Medium 20160401

For several decades sociologists have speculated that the performance of firms and other organizations depends as much on the networks of information flow between employees as on the formal structure of the organization [1, 2].

This argument makes intuitive sense, but until recently it has been extremely difficult to test using data.

Historically, employee data has been collected mostly in the form of surveys, which are still the gold standard for assessing opinions, but reveal little about behavior such as who talks to whom. Surveys are also expensive and time consuming to conduct, hence they are unsuitable for frequent and comprehensive snapshots of the state of a large organization.

Thanks to the growing ubiquity of productivity software, however, this picture is beginning to change. Email logs, web-based calendars, and co-authorship of online documents all generate digital traces that can be used as proxies for social networks and their associated information flows. In turn, these network and activity data have the potential to shed new light on old questions about the performance of teams, divisions, and even entire organizations.

Recognizing this opportunity, my colleagues Jake Hofman, Christian Perez, Justin Rao, Amit Sharma, Hanna Wallach, and I — in collaboration with Office 365 and Microsoft’s HR Business Insights unit — have embarked on a long-term project: the Organizational Spectroscope.

The Organizational Spectroscope combines digital communication data, such as email metadata (e.g., time stamps and headers), with more traditional data sources, such as job titles, office locations, and employee satisfaction surveys. These data sources are combined only in ways that respect privacy and ethical considerations. We then use a variety of statistical modeling techniques to predict and explain outcomes of interest to employees, HR, and management.

Predicting team satisfaction

To illustrate the potential of these new data and methods, we analyzed the aggregate email activity patterns of teams of US-based Microsoft employees to predict their responses to an annual employee satisfaction survey. To protect individual employee privacy only email metadata was used (i.e., no content) and all identifiers were encrypted. Email activity and survey responses were aggregated to the manager level, where only managers with at least five direct reports were included, and only these aggregated results were analyzed. Our predictions therefore apply only to teams of employees who share the same manager, not to individuals.

We focused on three survey questions: did teams have confidence in the overall effectiveness of their managers, did they think that different groups across the company collaborated effectively, and were they satisfied with their own work — life balance?

We started by examining the data and found that that the vast majority of teams were pretty happy. Although this result is encouraging, as a practical matter HR managers are less interested in the large majority of happy teams than in identifying the small minority of unhappy teams. After all, it is the latter group on which HR needs to focus its resources. Rather than trying to predict the satisfaction level of every team, therefore, we focused on predicting just the teams in the bottom 15% — i.e., the least satisfied teams.

We considered two statistical models: first, a simple linear logistic regression model of the type that is widely used in quantitative social science; and second, a more complicated model from machine learning called a random forest [3]. Although random forests are generally less interpretable than standard regression models, making them less well suited to the explanation tasks typically found in the social sciences, they can capture nonlinearities and heterogeneous effects that linear models ignore and therefore often perform better at prediction tasks.

In our case the random forest performed much better: if it predicted that a team was in the bottom 15%, it was correct (across all three questions) between 80% and 93% of the time; in contrast, the linear model was correct at best 27% of the time. Critically, a ‘‘baseline’’ model that used only data on respondents’ position and level in the company — i.e., no email activity features — performed between 20 and 40 percentage points worse. In other words, the email activity data added large and significant value over and above the kind of data that HR managers already have (see Table 1).

Table 1. Precision of the random forest model (column 3) compared with a standard logistic model (column 1). Column 2 is for a random forest model that does not include email activity, but does include other features such as job category (engineer, product manager, sales, etc.) and level.

Table 1 also shows the particular features of email activity that were most predictive of low satisfaction. For work — life balance, it was the fraction of emails sent out of working hours: more is worse. For managerial satisfaction, it was manager response time: slower is worse. And for perceptions of company-wide collaboration, it was the size of the manager’s email network: smaller is worse.

At first glance these findings may seem unsurprising, but this reaction misses the point. To see why, consider work — life balance. Although it makes sense that sending an unusual volume of email outside of normal working hours would correspond to low satisfaction, it would have made equal sense that satisfaction was related to the overall volume of email sent or received, or to the relative distribution of email over days of the week. But none of these other factors were useful for predicting low satisfaction. The point, therefore, is not so much about finding results that are surprising and counterintuitive, but about ruling out all the plausible, intuitive explanations that are not in fact correct.

Another non-obvious finding is that different types of teams had different thresholds for what counted as a “bad” volume of out-of-hours email. The number of out-of-hours emails that predicted an unhappy sales team, for example, was different from that of an unhappy engineering team. Again, this result isn’t surprising (once you know it), but it would have been difficult to guess in advance. This result also highlights the advantages of using a complicated model over a simple one: although in general we believe that, all else equal, simple models are better, when effects are highly context-dependent, complex models can shine.

Lessons for managers and for science

Insights like these are of immediate interest to both employees and managers. In particular, because predictions based on email sending behavior can be made in real time, HR can obtain more timely feedback than surveys allow. Moreover, modern statistical modeling approaches such as ours can help managers in complex situations where many different factors could be at play — e.g., by showing which of many plausible explanations are supported by the evidence, and by cautioning against “one size fits all” solutions. Finally, employees could also benefit from tools that highlight help them quantify their work activity in the same way that personal fitness trackers help them quantify physical activity.

More generally, our results show how the combination of novel sources of digital data and modern machine learning methods — that is, computational social science — can yield insights that would not be available with traditional data sources and methods. Over time, we hope to expand this approach from the specific case of predicting team satisfaction to a much wider range of questions regarding teams, divisions, and even entire organizations.

Finally, it is worth emphasizing that deriving these kinds of insights requires a lot of care. To perform our analysis, we combined three datasets — email activity, the org chart, and poll results— that were collected in different ways at different times by different people. Joining these data sets in a manner that respected privacy and ethical concerns required significant effort and cooperation across teams, which in turn required us to clearly specify, and justify, our substantive research questions and goals. Likewise the realization that we needed to focus on only the least satisfied teams required us to think carefully about the structure of the data and about our research questions. For all the excitement about “big data,” in other words, computational social science works well only when powerful computation is matched with careful social science.

1. Burns, T. and G.M. Stalker, The management of innovation. 1961, London: Tavistock Publications. v, 269.
2. Lawrence, P.R. and J.W. Lorsch, Organization and environment; managing differentiation and integration. 1967, Boston: Division of Research Graduate School of Business Administration Harvard University. xv, 279.
3. Breiman, L., Random forests. Machine learning, 2001. 45(1): p. 5–32.

Segura, Jerome – Canadian hospital serves ransomware via hacked website – 20160321

Segura, Jerome – Canadian hospital serves ransomware via hacked website – 20160321

Ransomware attacks have made a lot of headlines in the past year with several high-profile cases, including that of the Hospital in Los Angeles which had its data encrypted and ended up paying the ransom to get it back. Recently, the Ottawa hospital in Canada was also hit but able to contain a ransomware attack.

We discovered the website of another Canadian hospital had been compromised to actually spread ransomware to its visitors: staff, patients and families being the most likely to have visited the site. Norfolk General Hospital, based in Ontario, became a teaching facility for McMaster University’s Faculty of Health Sciences in 2009.

The web portal is powered by the Joomla CMS, running version 2.5.6 (latest version is 3.4.8) according to a manifest file present on their server. Several vulnerabilities exist for this outdated installation, which could explain why the site has been hacked.

Our honeypots visited the hospital page and got infected with ransomware via the Angler exploit kit. A closer look at the packet capture revealed that malicious code leading to the exploit kit was injected directly into the site’s source code itself.

Like many site hacks, this injection is conditional and will appear only once for a particular IP address. For instance, the site administrator who often visits the page will only see a clean version of it, while first timers will get served the exploit and malware.

Flow

The particular strain of ransomware dropped here is TeslaCrypt which demands $500 to recover your personal files it has encrypted. That payment doubles after a week.

Insecure web platforms still prevalent

We still see a large number of websites that are running outdated server-side software, namely WordPress and Joomla websites. Along with malvertising, hacked websites are the largest vehicle for new malware infections.

Common reasons for not updating a website include lack of resources, fear of breaking existing applications or simply forgetting to keep up with security patches.The truth of the matter is that any outdated or poorly secured website is simply a sitting duck waiting to be taken over via automated scanners before getting leveraged for spam, phishing or malicious redirections, just to name a few.

We contacted the Norfolk hospital and eventually were able to speak with their IT staff. We shared the information we had (screenshots, network packet capture) and told them about the ransomware payload we collected when we reproduced the attack in our lab. We were told that they were working on upgrading their version of Joomla with their hosting provider.

Ransomware in Canada

This particular attack prompted us to look at the state of ransomware in Canada. Since January of this year, Malwarebytes Anti-Malware has detected over ten thousand instances of ransomware affecting Canadians while many more were already proactively proactively blocked by our Anti-Exploit or Anti-Ransomware Beta products.

Here’s a break down for the top 10 Canadian cities most affected by ransomware according to our telemetry:

Toronto
Ottawa
Montreal
Markham
Calgary
Vancouver
London
Edmonton
Winnipeg
Saint Catharines
It is better to be safe than sorry when it comes to ransomware. Back up your files at least once a week and if possible keep those backups on an external media. Prevent infections by using proper security hygiene and multiple layers of defense.

Unfortunately, just as there are insecure websites, there are even more personal computers that are vulnerable and end up being infected. Because backups are seldom performed, a lot of users will find themselves in difficult situations where they desperately need their data back and feel forced to pay the ransom.

Sadly, those combined factors explain why ransomware is so prevalent and why new families and copycats are emerging all the time. Online criminals are fully tapping into this new haven that is extortionware.

Zaino, Jennifer – Ontology Plays a Part in United Nations Sustainable Development Goals Project – 20160303

Zaino, Jennifer – Ontology Plays a Part in United Nations Sustainable Development Goals Project – 20160303

How do you define “basic services”? What’s the difference between them and “essential services”? What is meant by terms like “natural capital”, “raw material” or “essential medicines”? How do these all fit into an ontology?

The truth is that there often aren’t universally accepted or precise definitions for terms like these – or even more simple ones, like “forest” – as they relate to their use in the United Nations Environment Program’s (UNEP) Sustainable Development Goals Project. The Sustainable Development Goals (SDG) – successors to the UN’s Millennial Development Goals, which expired last year – are a set of 17 goals and 169 targets to be achieved by 2030 to promote human prosperity worldwide while protecting the environment and addressing climate change.

For example, within the goal of ending poverty in all its forms everywhere, targets include reducing at least by half the proportion of men, women, and children who live in poverty in all its dimensions (according to national definitions); to implement nationally appropriate social protection systems and measures for all; and, to reduce poor people’s exposure and vulnerability to climate-related extreme events and other economic, social, and environmental shocks and disasters. Each goal’s targets will have one or more indicators, which are linked to specific data points that UN statisticians and the general public can monitor to assess progress on that issue.

For the project to come together in a way that ensures data quality and transparency and sets the stage to enable more accurate analysis and measurement of progress, the Sustainable Development Goals Interface Ontology (SDGIO) is being developed to represent the various facets of the SDGs.

An ontology is a structured set of terms and logical relations that represents not only the data, but what the data is about, says Mark Jensen, who is working on the UNEP ontology effort as a consultant. Jensen is pursuing his Ph.D. in the Department of Biomedical Informatics in the Jacobs School of Medicine and Biomedical Sciences at the University at Buffalo.

“An important distinction that well-made ontologies maintain is one between the information stored in databases and the entities out in the world that are described, measured or designated by that information,” he says.

The SDGIO project employs an approach in creating ontologies based in ontological realism, developed in large part by Barry Smith. Smith, who brought the UN effort to Jensen’s attention, is SUNY Distinguished Professor in the Department of Philosophy and Director of the National Center for Ontological Research at the university.

One use of ontologies in the UN project is to make it possible to intelligently tag incoming data to ensure that users are able to discover and better understand the information they are seeking, even when target and indicator data points intersect across domains and when there are inconsistencies in how member states define data points like “basic services” or “safe access” or, yes, even “forest.” For example, some definitions of “forest” will include palm tree plantations, while others do not, which can potentially impact data that relates to forest acreage, and subsequently, any analysis that occurs when data is aggregated across countries and regions with varying conceptions of what qualifies as a forest.

“As ontologists, we create semantic models that represent knowledge in particular domains of inquiry,” says Jensen. A UNEP ontology will provide a model to represent knowledge that’s relevant to the SDGs with more precision and better consistency, and that will provide a better way of integrating information used in monitoring the status of how various targets and goals are being addressed around the world, he explains.

Steps to the Ontology

Jensen is working closely on the SDGIO project with Pier Luigi Buttigieg, a post-doctoral researcher at the Alfred Wegener Institute for Polar and Marine Research. The lead of the Environment Ontology project (ENVO), Buttigieg was invited to participate in an UNEP-led meeting on integrated measures for global monitoring of the SDG process in 2014, Jensen relates. The value of the ontology in promoting integration and interoperability was recognized at that time by UNEP Chief Scientist Jacqueline McGlade – who is directing the SDG project with Ludgarde Coppens, UNEP head of the SDG Information and Knowledge Management Unit – and other representatives. This, Jensen says, resulted in the formation of a team of ontologists to create the SDGIO. “Pier continues to play a leading role in SDGIO’s evolution and is enhancing ENVO to help meet its aims,” says Jensen.

The aim of creating a better, more uniform approach to representing the data doesn’t mean changing the way member states conceptualize their own understandings of certain terms. But it does mean creating a way to represent the diversity of definitions and make that diversity of usage more transparent to people looking for data that is relevant to indicators. In addition to that, there needs to be a way to show how disparate data is linked together through a variety of common themes that cuts across multiple goals and targets.

An ontology enters the picture because of the advantages it affords of being a precise way to go about defining terms In hierarchical fashion and establishing relationships and formalized links between the lower-level terms in that hierarchy, he says. A top-level general definition for the term “forest,” then, can encapsulate all its different conceptualizations, with the variations between definitions represented as different lower-tier species or versions of what qualifies as a forest. There’s a great deal of unpacking of the semantics, or meaning, behind each indicator that’s required to facilitate the consistency users need to measure progress toward targets by making clear the links between various data and indicators, too. For instance, an indicator that is about the proportion of a population living in households with access to basic services has to account for all kinds of data that could relate to it, including what qualifies as a household and how that differentiates from living in a slum or informal settlement.

Jensen says the team will be finalizing the first phase of the ontology this spring, which will be implemented on the portal UNEPLive:

“An important aspect of the workflow is to elicit the feedback of relevant domain experts to guide their efforts in refining the semantics to better reflect the various domains surrounding the SDGs, such as legal entities, social policies, economic systems, equity and human rights, ecosystems and biodiversity, infrastructure, public health and education,” Jensen says.

This first phase includes the discovery and creation of all the needed terms and definitions and their formal implementation in OWL as an ontology. The team will reuse existing efforts by other groups developing ontologies, including those developed by ENVO (for the environment), CHEBI (for chemicals), OBI (for biomedical investigation), and PCO (for population and communities), and the complete semantics will grow over time as new ontologies are formed to address many of the domains that work on SDGIO has revealed a need for. For example, no ontology yet exists for human rights or financial measures, Jensen explains:

“SDGIO is truly an ‘interface.’ Not only is there a need to interface the data with the goals, targets and indicators, but also to interface the growing community of domain-specific ontologies that are and will be relevant to the SDGs.”

UN statisticians will tag the data using a few terms from the ontology (in addition to metadata such as provenance, geographic location, and so on). The ontology’s value for helping to enhance the discoverability of data should become clear in a few months as users go into the portal and type something like “access to basic services” or simply “basic services.” “We want to provide the ability for them to find data related to basic services in all contexts, not just for one particular indicator,” Jensen says.

Alternately, users might go in looking for data related to “essential services” but be served up data tagged as “basic services” because the two categories overlap very closely and often the distinction is hard to maintain. “We make the links between these terms and formalize that in the ontology so that if you search for one you can also find information tagged with the other,” he says. Researchers can drill down to determine whether information aligned with the other data tag fits their assessment requirements and use it or not, as they see fit.

Users also will be able to leverage the ontology for visualizing terms and the relationships between them. Along with those definitions they’ll find editor notes and comments about variations in usage. Says Jensen, “Hopefully they’ll utilize not just the discovery/search feature the ontology facilitates but also pay attention to the mapping and semantics it affords and the extra annotations we can add.”

Pepinsky, Tom – Simple models for complex politics – 20160304

Pepinsky, Tom – Simple models for complex politics – 20160304

Politics is complex. For scholars of comparative politics who study domestic politics in an increasingly globalized world, understanding the interactions among local, national, transnational, regional, and global forces is essential. So how should we proceed? One view is that grasping complexity means discarding simple theories and spare models of politics that do not reflect the complexity that we know exists. The position is intuitively appealing: complex problems require complex tools.

There is, however, another view. My colleague Andrew Little and I have recently finished a new paper on formal theory in comparative politics entitled “Simple Formal Models in Comparative Politics” (PDF). It is written as part of a dialogue on the future of comparative politics, and responds (in part) to work by Philippe Schmitter (see e.g. here) in which complexity[*], multi-level politics, and the dangers of simplifying assumptions figure prominently. Part of the paper is a clarification of the state of formal theory in comparative politics. We show that formal theory still occupies a relatively small part of the work being done in comparative politics, and that there is scant evidence that this is going to change any time soon. We also comment on some common beliefs about what models and assumptions are for which, sadly, remain all too common in the discipline.

But more interesting and broadly relevant is what comes next. We argue that in a world of complex interactions, simplification—in formal theory as in other kinds of theorizing—is a virtue, not a vice. We explain why in detail in the paper, but at root is the fact that theories are always simplifications, and descriptive accuracy is but one criterion by which a theory ought to be judged. We also suggest that professional incentives lead modelers to create formal models that are more complicated than they need to be. Our suggestions for how simple models of politics (formal or otherwise) might join together with the “complexity-embracing” modes of research is a nice parallel to recent contributions by Gehlbach (PDF) and Lorentzen et al (PDF). Our perspective on theory-as-simplification also parallels Healy’s colorful reflections on “nuance” (PDF).

So yes, the politics is complex, but this does not mean our theories must be also. Instead, we need multiple just-simple-enough theories, and continuous collaboration with case experts and other empiricists to know what “just-simple-enough” means.

Note

[*] Without speaking for my coauthor, I doubt that this is a particularly revolutionary idea when the term “complex interdependence” is nearly a century old, and prominent scholars have been asking questions like “is the traditional distinction between international relations and domestic politics dead?” longer than I have been alive (see Gourevitch 1978).

Installing tools for d3.geo – 20160306

In Let’s make a map, Mike Bostock describes how to make a modest map from scratch using D3 and TopoJSON. Here we detail how to install the main tools on our CentOS 6x server.

Installing Tools

Geographic data files are almost always too large for manual cleanup or conversion. Fortunately, there’s a vibrant geo open-source community, and many excellent free tools to manipulate and convert between standard formats.

GDAL

The big multitool to know is the Geospatial Data Abstraction Library. Commonly referred to as GDAL, it includes the OGR Simple Features Library and the ogr2ogr binary we’ll use to manipulate shapefiles. There are official GDAL binaries for a variety of platforms – our hosted service runs on CentOS 6x.

The Enterprise Linux GIS (ELGIS) effort provides RPMs of various GIS applications, including GDAL, for CentOS and other Enterprise Linux derivatives. To upload the release RPM for CentOS 6x:

sudo rpm -Uvh http://elgis.argeo.org/repos/6/elgis-release-6-6_0.noarch.rpm

To install GDAL:

sudo yum install -y gdal

… but this generates a lot of dependency errors. In the end, the most significant issue we had to address was GDAL requirement for armadillo-3x instead of the newer (and not backward-compatible vis-a-vis GDAL) armadillo-4x.

We tracked down a copy of armadillo-3.800.2-1.el6.src.rpm, uploaded it to our /root directory, and installed:

sudo rpm -Uvh armadillo-3.800.2-1.el6.x86_64.rpm

… and then we were able to install the GDAL RPM package, as above.

Miscellaneous things we did along the way to installing GDAL

While we’re uncertain of their ultimate significance, we installed/compiled some resources that are worth noting:

Some of the missing dependencies that were flagged up to us were addressed by installing two packages:

su -c 'rpm -Uvh http://mirror.centos.org/centos/6/os/x86_64/Packages/atlas-3.8.4-2.el6.x86_64.rpm

su -c 'rpm -Uvh http://dl.fedoraproject.org/pub/epel/6//x86_64/arpack-3.1.3-1.el6.x86_64.rpm'

We also compiled proj4 and geos:

wget ftp://ftp.remotesensing.org/proj/proj-4.6.0.tar.gz
tar -zxvf proj-4.6.0.tar.gz
cd proj-4.6.0
./configure
sudo make install

cd ..
wget http://geos.refractions.net/downloads/geos-3.0.0.tar.bz2
tar -jxvf geos-3.0.0.tar.bz2
cd geos-3.0.0
./configure
sudo make install
# add lib path to ld.so.conf file
sudo vi /etc/ld.so.conf
# add this line
/usr/local/lib
sudo /sbin/ldconfig
# Add lib path to ld.so.conf file
sudo vi /etc/ld.so.conf
# add this line
/usr/local/lib
sudo /sbin/ldconfig

TopoJSON

Next you’ll need the reference implementation for TopoJSON, which requires Node.js. Fortunately, we’d already arranged for our host to install Node.js. To install TopoJSON:

npm install -g topojson

… which throws a warning: {several modules} requires inherits@’2′ but will load – even after we install pm2:

npm install pm2 -g

[note: when we were trying to merge shp files, threw error that someone suggested would be fixed by uninstalling topojson and re-installing with sudo -H npm install -g topojson]

To check the installation:

which ogr2ogr
# prints /usr/local/bin/ogr2ogr
which topojson
# prints /usr/local/bin/topojson

Health Quality Ontario – Hospital Quality Improvement – Resources

Watts, D – How small is the world, really? – Medium 20160210

Watts, D – How small is the world, really? – Medium 20160210
Last week’s finding by a team of data scientists at Facebook that everyone in the social network is connected by an average of 3.5 “intermediaries” has renewed interest in the longstanding “Six Degrees of Separation” hypothesis: that everyone in the world is connected by some short chain of acquaintances. Not surprisingly, the attention has focused on the plausible assertion that online social networks like Facebook have made the world smaller: that whatused to be six degrees is now almost half that. But really what it has revealed is how little we understand this intriguing phenomenon and what it might mean for our world.

This “small world” hypothesis, as it is known in sociology, has been percolating in popular culture for a long time. Almost a century ago the Hungarian poet Frigyes Karinthy wrote a short story called “Chain Links” in which he claimed he could reach anyone in the world, whether a Nobel Prize winner or a worker in a Ford auto factory, through a series of no more than five intermediaries. Subsequently, writers like Jane Jacobs, John Guare, andMalcolm Gladwell have periodically reinvigorated the idea with their own colorful characters and fantastical speculations about who really runs the world.

But arguably no one has had more impact on the question of how small the world is than Stanley Milgram, a Harvard psychologist who in the 1960s conducted an ingenious experiment to test it (Milgram is even more famous for another experiment of his, on obedience to authority, but that’s for another day). In brief, Milgram chose a single person, an acquaintance of his who was a stockbroker living in Sharon Mass, just outside of Boston, to be the “target” of the experiment. In addition he chose roughly 300 others — 100 from Boston itself and the other 200 from Omaha Nebraska, which Milgram figured was about as far away from Boston, socially and geographically, as one could get within the US.

Milgram then sent these 300 subjects special packets containing a good deal of information about the target — his name, address, occupation, etc. — and also instructions that they were to try to get the packet to him. But there was a catch: they could only send the packet to him if they knew him personally, meaning on a first-name basis. In the overwhelmingly likely event that that they did not, they were instead to send to someone they did know on a first name basis and who was closer to the target than they were themselves. These new participants would then get the same packet with the same instructions, and the process would repeat until — hopefully — some of the packets reached the target.

Milgram’s question then was: for successfully delivered packets, how long would the chains be? Curiously, before he ran the experiment Milgram asked lots of people to guess the answer. Many assumed it wasn’t possible while others figured it would take hundreds of steps. So when Milgram found that not only did 64 packets, roughly one fifth of the initial sample, reached the target, but that the average length of the successful chains was just 6, he knew it would surprise many people.

In many ways, it still does. Although the phrase “Six Degrees of Separation” has become a cliché, when pressed many people still find it difficult to imagine how they could really reach anyone — not just someone like them or someone near to them, but anyone at all in the whole world — in something like six steps. Understandably then, the Facebook result also attracted some resistance: “Facebook is an unrepresentative sample of the population;” “Facebook friends aren’t real friends” and so on. But although these critiques may have merit, they miss the point. In reality, the 3.5 number is simply incomparable to Milgram’s 6 for three reasons.

First, the number 3.5 counts intermediaries not degrees of separation. If I am “one degree” from someone I know them directly; there are zero intermediaries between me and them. Likewise, there is one intermediary between me and my “two degree” neighbors, and so on. In general, therefore, an average of 3.5 intermediaries corresponds to 4.5 degrees of separation, which is almost exactly what Facebook itself found when it performed asimilar exercise a few years ago. Conversely, Milgram’s six degrees result corresponds to five intermediaries, which is actually the number he reported in his original paper with Jeffery Travers. So already the difference is one less than it appears.

Second, though, Milgram’s experiment was a subtly but importantly different test than the one run by Facebook. Whereas the latter measured the length of the shortest possible path between two people — by exhaustively searching every link in the underlying Facebook graph — the former is simply the shortest path that ordinary people could find given very limited information about the underlying social network. There are, in other words, two versions of the small-world hypothesis — the “topological” version, which refers only to underlying network structure, and the “algorithmic” version, which refers to the ability of people to search this underlying structure. From these definitions, it follows that algorithmic (search) paths cannot be shorter than topological paths and are almost certainly longer. Saying that the world has gotten smaller because the shortest topological path length is 4.5 not 6 therefore makes no sense — because the equivalent number would have been smaller in Milgram’s day as well.

Finally, the number 6 is also in some respects too small. As has been pointed out many times since Milgram’s experiment, only about 20% of the letters made it to their target. More importantly, these letters were almost certainly on shorter paths than the ones that didn’t make it, meaning that estimates of path length that don’t take into account the missing data are almost certainly biased downwards. Fortunately it is possible to correct for this bias using standard statistical methods. In a 2009 paper my colleagues and I performed exactly this analysis both on Milgram’s original data and also on our data from a similar — but much larger — experiment that we had conducted ourselves in 2003.

Remarkably we found that after the correction, both experiments yielded similar results: the median shortest path was 7, meaning that 50% of chains should complete in 7 or fewer steps while the other 50% would be longer. Many people find this result surprising because it seems so clear that the world has gotten smaller in the last 50 years. Yet this apparent stability is exactly what one would predict from my early theoretical work with Steven Strogatz back in the late 1990’s. In a nutshell what we showed is that it is easy to turn a “large” world into a “small” one, just by adding a small fraction of random, long-range links, reminiscent of Mark Granovetter’s famous “weak ties.” The flip side of our result, however, is that once the world has already gotten small — as it was already by the 1960’s — it is extremely hard to make it smaller. Obviously Facebook did not exist in 2003 so possibly since then something has indeed changed. But I suspect that the difference will be small.

Why does any of this matter? There are three reasons. First, the two versions of the small-world hypothesis — topological and algorithmic — are relevant to different social processes. The spread of a sexually transmitted disease along networks of sexual relations, for example, does not require that participants have any awareness of the disease, or intention to spread it; thus for an individual to be at risk of acquiring an infection, he or she need only be connected in the topological sense to existing infectives. On the contrary, individuals attempting to “network” — in order to locate some resources like a new job or a service provider — must actively traverse chains of referrals, and thus must be connected in the algorithmic sense. Depending on the application of interest, therefore, either the topological or algorithmic distance between individuals may be more relevant — or possibly both together. Second, whereas the topological hypothesis has been shown to apply essentially universally, to networks of all kinds, the algorithmic hypothesis is largely (although not exclusively) concerned with social networks in which human agents make decisions about how to direct messages. And third, whereas the topological version is supported by an overwhelming volume of empirical evidence — hundreds of studies, if not thousands — have found that nodes in even the very largest known networks are connected by short paths, the practical difficulty of running “small-world” experiments of the sort that Milgram conducted in the 1960s has meant that much less is known about the algorithmic version.

On this last point, for example, our 2009 analysis also found evidence that some of the longer paths could be much longer than the median, adding weight to the skeptics’ claims that in spite of the small-world phenomenon, some people remain socially isolated. Given the importance of social networks in determining life outcomes, it would be extremely interesting and useful to understand better who these people are and why they are isolated. Is it something to do with their underlying networks or is it that their search strategies are somehow less effective? Could it be, as my coauthors and Ispeculated many years ago, some kind of self-fulfilling prophecy, in which theperception of social isolation discourages one from searching one’s network, and that the resulting lack of success reinforces the original perception of isolation? Answering these questions would require new experiments that are only now just becoming possible. But the answers would not only be of academic interest — they could also potentially help many people access currently inaccessible reserves of “social capital” thereby improving their lives. Far from being settled, the small-world problem still has much to teach us about the world, and ourselves.

Simpson’s Paradox and other possible anomalies with 3+ way contingency tables

t – A command-line power tool for Twitter – sferik

A command-line power tool for Twitter.

The CLI takes syntactic cues from the Twitter SMS commands, but it offers vastly more commands and capabilities than are available via SMS.

Installation

First, make sure you have Ruby installed.

On Windows,

  • Install Ruby with RubyInstaller to c:/ruby21
  • Download the Development Kit: DevKit-mingw64-32-4.7.2-20130224-1151-sfx.exe
  • Run the Development Kit .exe to extract it to c:/ruby21 (permanent).
  • In the Ruby command line interface, cd to c:/ruby21/mingw
  • Run ruby dk.rb init and ruby dk.rb install to bind the Development Kit (mingw) to the Ruby installation in your path.

Once Ruby and the Development Kit are installed, install the t gem package:

gem install t

Configuration

Twitter API v1.1 requires OAuth for all of its functionality, so you’ll need a registered Twitter application. If you’ve never registered a Twitter application before, it’s easy! Just sign-in using your Twitter account and then fill out the short form athttps://apps.twitter.com//new. If you’ve previously registered a Twitter application, it should be listed athttps://apps.twitter.com/. Once you’ve registered an application, make sure to set your application’s Access Level to “Read, Write and Access direct messages”, otherwise you’ll receive an error that looks like this:

Error processing your OAuth request: Read-only application cannot POST

A mobile phone number must be associated with your account in order to obtain write privileges. If your carrier is not supported by Twitter and you are unable to add a number, contact Twitter using https://support.twitter.com/forms/platform, selecting the last checkbox. Some users have reported success adding their number using the mobile site,https://mobile.twitter.com/settings, which seems to bypass the carrier check at the moment.

Now, you’re ready to authorize a Twitter account with your application. To proceed, type the following command at the prompt and follow the instructions:

t authorize

This command will direct you to a URL where you can sign-in to Twitter, authorize the application, and then enter the returned PIN back into the terminal. If you type the PIN correctly, you should now be authorized to use t as that user. To authorize multiple accounts, simply repeat the last step, signing into Twitter as a different user.

NOTE: If you have problems authorizing multiple accounts, open a new window in your browser in incognito/private-browsing mode and repeat the t authorize steps. This is apparently due to a bug in twitter’s cookie handling.

You can see a list of all the accounts you’ve authorized by typing the command:

t accounts

The output of which will be structured like this:

sferik
  UDfNTpOz5ZDG4a6w7dIWj
  uuP7Xbl2mEfGMiDu1uIyFN
gem
  thG9EfWoADtIr6NjbL9ON (active)

Note: One of your authorized accounts (specifically, the last one authorized) will be set as active. To change the active account, use the set subcommand, passing either just a username, if it’s unambiguous, or a username and consumer key pair, like this:

t set active sferik UDfNTpOz5ZDG4a6w7dIWj

Account information is stored in a YAML-formatted file located at ~/.trc.

Note: Anyone with access to this file can impersonate you on Twitter, so it’s important to keep it secure, just as you would treat your SSH private key. For this reason, the file is hidden and has the permission bits set to 0600.

Usage Examples

Typing t help will list all the available commands. You can type t help TASK to get help for a specific command.

t help

Update your status

t update "I'm tweeting from the command line. Isn't that special?"

Note: If your tweet includes special characters (e.g. !), make sure to wrap it in single quotes instead of double quotes, so those characters are not interpreted by your shell. If you use single quotes, your Tweet obviously can’t contain any apostrophes unless you prefix them with a backslash \:

t update 'I\'m tweeting from the command line. Isn\'t that special?'

Retrieve detailed information about a Twitter user

t whois @sferik

Retrieve stats for multiple users

t users -l @sferik @gem

Follow users

t follow @sferik @gem

Check whether one user follows another

t does_follow @ev @sferik

Note: If the first user does not follow the second, t will exit with a non-zero exit code. This allows you to execute commands conditionally. For example, here’s how to send a user a direct message only if they already follow you:

t does_follow @ev && t dm @ev "What's up, bro?"

Create a list for everyone you’re following

t list create following-`date "+%Y-%m-%d"`

Add everyone you’re following to that list (up to 500 users)

t followings | xargs t list add following-`date "+%Y-%m-%d"`

List all the members of a list, in long format

t list members -l following-`date "+%Y-%m-%d"`

List all your lists, in long format

t lists -l

List all your friends, in long format, ordered by number of followers

t friends -l --sort=followers

List all your leaders (people you follow who don’t follow you back)

t leaders -l --sort=followers

Mute everyone you follow

t followings | xargs t mute

Unfollow everyone you follow who doesn’t follow you back

t leaders | xargs t unfollow

Unfollow 10 people who haven’t tweeted in the longest time

t followings -l --sort=tweeted | head -10 | awk '{print $1}' | xargs t unfollow -i

Twitter roulette: randomly follow someone who follows you (who you don’t already follow)

t groupies | shuf | head -1 | xargs t follow

Favorite the last 10 tweets that mention you

t mentions -n 10 -l | awk '{print $1}' | xargs t favorite

Output the last 200 tweets in your timeline to a CSV file

t timeline -n 200 --csv > timeline.csv

Start streaming your timeline (Control-C to stop)

t stream timeline

Count the number of official Twitter engineering accounts

t list members twitter/engineering | wc -l

Search Twitter for the 20 most recent Tweets that match a specified query

t search all "query"

Download the latest Linux kernel via BitTorrent (possibly NSFW, depending on where you work)

t search all "lang:en filter:links linux torrent" -n 1 | grep -o "http://t.co/[0-9A-Za-z]*" | xargs open

Search Tweets you’ve favorited that match a specified query

t search favorites "query"

Search Tweets mentioning you that match a specified query

t search mentions "query"

Search Tweets you’ve retweeted that match a specified query

t search retweets "query"

Search Tweets in your timeline that match a specified query

t search timeline "query"

Search Tweets in another user’s timeline that match a specified query

t search timeline @sferik "query"

Features

  • Deep search: Instead of using the Twitter Search API, which only goes back 6-9 days, t search fetches up to 3,200 tweets via the REST API and then checks each one against a regular expression.
  • Multi-threaded: Whenever possible, Twitter API requests are made in parallel, resulting in faster performance for bulk operations.
  • Designed for Unix: Output is designed to be piped to other Unix utilities, like grep, comm, cut, awk, bc, wc, and xargs for advanced text processing.
  • Generate spreadsheets: Convert the output of any command to CSV format simply by adding the --csv flag.
  • 95% C0 Code Coverage: Well tested, with a 2.5:1 test-to-code ratio.

Using T for Backup

@jphpsf wrote a blog post explaining how to use t to backup your Twitter account.

t was also mentioned on an episode of the Ruby 5 podcast.

t was also discussed on an episode of the Ruby Rogues podcast.

If you discuss t in a blog post or podcast, let me know and I’ll link it here.

Relationship Terminology

There is some ambiguity in the terminology used to describe relationships on Twitter. For example, some people use the term “friends” to mean everyone you follow. In t, “friends” refers to just the subset of people who follow you back (i.e., friendship is bidirectional). Here is the full table of terminology used by t:

                           ___________________________________________________
                          |                         |                         |
                          |     YOU FOLLOW THEM     |  YOU DON'T FOLLOW THEM  |
 _________________________|_________________________|_________________________|_________________________
|                         |                         |                         |                         |
|     THEY FOLLOW YOU     |         friends         |        groupies         |        followers        |
|_________________________|_________________________|_________________________|_________________________|
|                         |                         |
|  THEY DON'T FOLLOW YOU  |         leaders         |
|_________________________|_________________________|
                          |                         |
                          |       followings        |
                          |_________________________|

Screenshots

TimelineList

Shell completion

If you’re running Zsh or Bash, you can source one of the bundled completion files to get shell completion for t commands, subcommands, and flags.

Don’t run Zsh or Bash? Why not contribute completion support for your favorite shell?

History

The twitter gem previously contained a command-line interface, up until version 0.5.0, when it was removed. This project is offered as a successor to that effort, however it is a clean room implementation that contains none of the original code.

History

Supported Ruby Versions

This library aims to support and is tested against the following Ruby implementations:

  • Ruby 1.9.3
  • Ruby 2.0.0
  • Ruby 2.1
  • Ruby 2.2

If something doesn’t work on one of these Ruby versions, it’s a bug.

This library may inadvertently work (or seem to work) on other Ruby implementations, however support will only be provided for the versions listed above.

If you would like this library to support another Ruby version, you may volunteer to be a maintainer. Being a maintainer entails making sure all tests run and pass on that implementation. When something breaks on your implementation, you will be responsible for providing patches in a timely fashion. If critical issues for a particular implementation exist at the time of a major release, support for that Ruby version may be dropped.

Troubleshooting

If you are running t on a remote computer you can use the flag –display-uri during authorize process to display the url instead of opening the web browser.

t authorize --display-uri

Agresti, A – Datasets

This site contains data sets that are not shown completely in text examples and exercises. (The numbering refers to the 3rd edition, 2013)

1. Horseshoe crab data set of Table 4.3

(Here y is whether a female crab has a satellite (1=yes, 0=no) and weight is in grams, rather than kg as in the text. Also, color has values 1-5 with 1=light; there were no crabs of color 1, so in the text, color was re-coded as color – 1 to give values 1, 2, 3, 4.)


color spine width satell weight y 
3  3  28.3  8  3050 1 
4  3  22.5  0  1550 0 
2  1  26.0  9  2300 1 
4  3  24.8  0  2100 0 
4  3  26.0  4  2600 1 
3  3  23.8  0  2100 0 
2  1  26.5  0  2350 0 
4  2  24.7  0  1900 0 
3  1  23.7  0  1950 0 
4  3  25.6  0  2150 0 
4  3  24.3  0  2150 0 
3  3  25.8  0  2650 0 
3  3  28.2  11 3050 1 
5  2  21.0  0  1850 0 
3  1  26.0  14 2300 1 
2  1  27.1  8  2950 1 
3  3  25.2  1  2000 1 
3  3  29.0  1  3000 1 
5  3  24.7  0  2200 0 
3  3  27.4  5  2700 1 
3  2  23.2  4  1950 1 
2  2  25.0  3  2300 1 
3  1  22.5  1  1600 1 
4  3  26.7  2  2600 1 
5  3  25.8  3  2000 1 
5  3  26.2  0  1300 0 
3  3  28.7  3  3150 1 
3  1  26.8  5  2700 1 
5  3  27.5  0  2600 0 
3  3  24.9  0  2100 0 
2  1  29.3  4  3200 1 
2  3  25.8  0  2600 0 
3  2  25.7  0  2000 0 
3  1  25.7  8  2000 1 
3  1  26.7  5  2700 1 
5  3  23.7  0  1850 0 
3  3  26.8  0  2650 0 
3  3  27.5  6  3150 1 
5  3  23.4  0  1900 0 
3  3  27.9  6  2800 1 
4  3  27.5  3  3100 1 
2  1  26.1  5  2800 1 
2  1  27.7  6  2500 1 
3  1  30.0  5  3300 1 
4  1  28.5  9  3250 1 
4  3  28.9  4  2800 1 
3  3  28.2  6  2600 1 
3  3  25.0  4  2100 1 
3  3  28.5  3  3000 1 
3  1  30.3  3  3600 1 
5  3  24.7  5  2100 1 
3  3  27.7  5  2900 1 
2  1  27.4  6  2700 1 
3  3  22.9  4  1600 1 
3  1  25.7  5  2000 1 
3  3  28.3  15 3000 1 
3  3  27.2  3  2700 1 
4  3  26.2  3  2300 1 
3  1  27.8  0  2750 0 
5  3  25.5  0  2250 0 
4  3  27.1  0  2550 0 
4  3  24.5  5  2050 1 
4  1  27.0  3  2450 1 
3  3  26.0  5  2150 1 
3  3  28.0  1  2800 1 
3  3  30.0  8  3050 1 
3  3  29.0  10 3200 1 
3  3  26.2  0  2400 0 
3  1  26.5  0  1300 0 
3  3  26.2  3  2400 1 
4  3  25.6  7  2800 1 
4  3  23.0  1  1650 1 
4  3  23.0  0  1800 0 
3  3  25.4  6  2250 1 
4  3  24.2  0  1900 0 
3  2  22.9  0  1600 0 
4  2  26.0  3  2200 1 
3  3  25.4  4  2250 1 
4  3  25.7  0  1200 0 
3  3  25.1  5  2100 1 
4  2  24.5  0  2250 0 
5  3  27.5  0  2900 0 
4  3  23.1  0  1650 0 
4  1  25.9  4  2550 1 
3  3  25.8  0  2300 0 
5  3  27.0  3  2250 1 
3  3  28.5  0  3050 0 
5  1  25.5  0  2750 0 
5  3  23.5  0  1900 0 
3  2  24.0  0  1700 0 
3  1  29.7  5  3850 1 
3  1  26.8  0  2550 0 
5  3  26.7  0  2450 0 
3  1  28.7  0  3200 0 
4  3  23.1  0  1550 0 
3  1  29.0  1  2800 1 
4  3  25.5  0  2250 0 
4  3  26.5  1  1967 1 
4  3  24.5  1  2200 1 
4  3  28.5  1  3000 1 
3  3  28.2  1  2867 1 
3  3  24.5  1  1600 1 
3  3  27.5  1  2550 1 
3  2  24.7  4  2550 1 
3  1  25.2  1  2000 1 
4  3  27.3  1  2900 1 
3  3  26.3  1  2400 1 
3  3  29.0  1  3100 1 
3  3  25.3  2  1900 1 
3  3  26.5  4  2300 1 
3  3  27.8  3  3250 1 
3  3  27.0  6  2500 1 
4  3  25.7  0  2100 0 
3  3  25.0  2  2100 1 
3  3  31.9  2  3325 1 
5  3  23.7  0  1800 0 
5  3  29.3  12 3225 1 
4  3  22.0  0  1400 0 
3  3  25.0  5  2400 1 
4  3  27.0  6  2500 1 
4  3  23.8  6  1800 1 
2  1  30.2  2  3275 1 
4  3  26.2  0  2225 0 
3  3  24.2  2  1650 1 
3  3  27.4  3  2900 1 
3  2  25.4  0  2300 0 
4  3  28.4  3  3200 1 
5  3  22.5  4  1475 1 
3  3  26.2  2  2025 1 
3  1  24.9  6  2300 1 
2  2  24.5  6  1950 1 
3  3  25.1  0  1800 0 
3  1  28.0  4  2900 1 
5  3  25.8  10 2250 1 
3  3  27.9  7  3050 1 
3  3  24.9  0  2200 0 
3  1  28.4  5  3100 1 
4  3  27.2  5  2400 1 
3  2  25.0  6  2250 1 
3  3  27.5  6  2625 1 
3  1  33.5  7  5200 1 
3  3  30.5  3  3325 1 
4  3  29.0  3  2925 1 
3  1  24.3  0  2000 0 
3  3  25.8  0  2400 0 
5  3  25.0  8  2100 1 
3  1  31.7  4  3725 1 
3  3  29.5  4  3025 1 
4  3  24.0  10 1900 1 
3  3  30.0  9  3000 1 
3  3  27.6  4  2850 1 
3  3  26.2  0  2300 0 
3  1  23.1  0  2000 0 
3  1  22.9  0  1600 0 
5  3  24.5  0  1900 0 
3  3  24.7  4  1950 1 
3  3  28.3  0  3200 0 
3  3  23.9  2  1850 1 
4  3  23.8  0  1800 0 
4  2  29.8  4  3500 1 
3  3  26.5  4  2350 1 
3  3  26.0  3  2275 1 
3  3  28.2  8  3050 1 
5  3  25.7  0  2150 0 
3  3  26.5  7  2750 1 
3  3  25.8  0  2200 0 
4  3  24.1  0  1800 0 
4  3  26.2  2  2175 1 
4  3  26.1  3  2750 1 
4  3  29.0  4  3275 1 
2  1  28.0  0  2625 0 
5  3  27.0  0  2625 0 
3  2  24.5  0  2000 0 

2. Teratology study data set of Table 4.7


 litter group n y 
 1  1 10 1   
 2 1 11 4   
 3 1 12 9   
 4 1 4 4     
 5 1 10 10   
 6 1 11 9
 7  1 9  9   
 8 1 11 11  
 9 1 10 10  
 10 1 10 7  
 11 1 12 12  
 12 1 10 9
 13 1 8  8  
 14 1 11  9 
 15 1 6  4   
 16 1  9 7  
 17 1 14 14  
 18 1 12 7
 19 1 11 9  
 20 1 13 8  
 21 1 14 5   
 22 1 10 10 
 23 1 12 10  
 24 1 13 8
 25 1 10 10 
 26 1 14 3  
 27 1 13 13  
 28 1 4 3   
 29 1  8  8  
 30 1 13 5
 31 1 12 12 
 32 2 10 1  
 33 2  3  1  
 34 2 13 1  
 35 2 12  0  
 36 2 14 4
 37 2  9  2 
 38 2 13 2  
 39 2 16  1  
 40 2 11 0  
 41 2  4  0  
 42 2 1  0
 43 2 12 0  
 44 3  8 0  
 45 3 11  1  
 46 3 14 0  
 47 3 14 1   
 48 3 11 0
 49 4  3 0   
 50 4 13 0  
 51 4 9   2  
 52 4 17 2  
 53 4 15 0   
 54 4 2 0
 55 4 14 1  
 56 4 8  0  
 57 4 6  0   
 58 4 17 0

3. Ray Allen data set for Exercise 4.13


1   0  4 
2   7  9
3   4 11
4   3  6
5   5  6
6   2  7
7   3  7
8   0  1
9   1  8
10  6  9
11  0  5
12  2  5
13  0  5
14  2  4
15  5  7
16  1  3
17  3  7
18  0  2
19  8 11
20  0  8
21  0  4
22  0  4
23  2  5
24  2  7

4. Rajon Rondo assists data set for Exercise 5.3


assists result * 1=win, last 9 observations are playoffs
 17 1 
  9 0 
 24 1 
 17 1 
 15 1 
 11 1 
 10 1 
 15 0 
 16 1 
 17 1 
 13 1 
  7 0 
 14 1 
 12 1 
 10 1 
 19 1 
 13 1 
 14 1 
  8 1 
 14 1 
  8 1 
 16 1 
 23 1 
  7 1 
  8 0 
 12 0 
 13 1 
 13 1 
 12 1 
  8 1 
 12 1 
  9 0 
 10 1 
  5 1 
  6 0 
 16 1 
 10 1 
 12 0 
  7 1 
 14 0 
 10 0 
 10 1 
  8 1 
 15 1 
  8 0 
 11 1 
 11 1 
 15 1 
 16 1 
  8 1 
  9 0 
  5 0 
  3 1 
  9 0 
  8 1 
  6 0 
  5 1 
 12 1 
 11 0 
  5 0 
  8 0 
 14 1 
  5 0 
 14 1 
 13 1 
  6 0 
 14 1 
  5 0 
 
  9 1 
  7 1 
 20 1 
 12 1 
  7 0 
 12 0 
 11 1 
  5 0 
  3 0 
;

5. Data on Italian credit cards, for Exercise 5.22


income n y
24  1  0  
34  7  1  
48  1  0  
70  5  3  
27  1  0  
35  1  1  
49  1  0  
79  1  0  
28  5  2  
38  3  1  
50  10  2  
80  1  0 
29  3  0  
39  2  0  
52  1  0  
84  1  0  
30  9  1  
40  5  0  
59  1  0  
94  1  0 
31  5  1  
41  2  0  
60  5  2  
120  6  6  
32  8  0  
42  2  0  
65  6  6  
130  1  1 
33  1  0  
45  1  1  
68  3  3     

6. Full data set for Table 6.2 on endometrial cancer grade


nv pi eh hg * standardized use nv2=(nv-0.5); pi2=(pi-17.3797)/9.9978; eh2=(eh-1.6616)/.6621;
datalines;
    0 13 1.64 0
    0 16 2.26 0
    0  8 3.14 0
    0 34 2.68 0
    0 20 1.28 0
    0  5 2.31 0
    0 17 1.80 0
    0 10 1.68 0
    0 26 1.56 0
    0 17 2.31 0
    0  8 2.01 0
    0  7 1.89 0
    0 20 3.15 0
    0 10 1.23 0
    0 18 1.27 0
    0 16 1.76 0 
    0 18 2.00 0
    0  8 2.64 1
    0 29 0.88 1
    0 12 1.27 1
    0 20 1.37 1
    1 38 0.97 1
    1 22 1.14 1
    1  7 0.88 1
    1 25 0.91 1
    1 15 0.58 1
    0  7 0.97 1
    0 28 1.50 0
    0 11 1.33 0
    0 19 2.37 0
    0 10 1.82 0
    0 10 3.13 0
    0 18 1.31 0
    0 14 1.92 0
    0 21 1.64 0
    0 11 2.01 0
    0 17 1.88 0
    0 25 1.93 0
    0 16 2.11 0
    0 19 1.29 0
    0 15 1.72 0
    0 33 0.75 0
    0 24 1.92 0
    0 48 1.84 1
    0 12 1.11 1
    0 19 1.61 1
    0  2 1.18 1
    1 22 1.44 1
    1 40 1.18 1
    1  5 0.93 1
    1  0 1.17 1
    0 21 1.19 1
    0 15 1.06 1
    0 29 2.02 0
    0 15 2.29 0
    0 12 2.33 0
    0  3 2.90 0
    0 20 1.70 0
    0 23 1.41 0
    0 12 2.25 0
    0 22 1.54 0
    0 42 1.97 0
    0 15 1.75 0
    0 13 2.16 0
    0 14 2.57 0
    0 19 1.37 0
    0 12 3.61 0
    0 13 2.04 0
    0 10 2.17 0
    0 12 1.69 1
    1 49 0.27 1
    0  6 1.84 1
    0  5 1.30 1
    0 17 0.96 1
    1 11 1.01 1
    1 21 0.98 1
    0  5 0.35 1
    1 19 1.02 1
    0 33 0.85 1 

7. Clinical trials data set of Table 6.9


 center treat response count 
 a 1 1 11 
 a 1 2 25  
 a 2 1 10  
 a 2 2 27
 b 1 1 16 
 b 1 2 4   
 b 2 1 22  
 b 2 2 10
 c 1 1 14 
 c 1 2 5   
 c 2 1 7   
 c 2 2 12
 d 1 1 2  
 d 1 2 14  
 d 2 1 1   
 d 2 2 16
 e 1 1 6  
 e 1 2 11  
 e 2 1 0   
 e 2 2 12
 f 1 1 1  
 f 1 2 10  
 f 2 1 0   
 f 2 2 10
 g 1 1 1  
 g 1 2 4   
 g 2 1 1   
 g 2 2 8
 h 1 1 4  
 h 1 2 2   
 h 2 1 6   
 h 2 2 1

8. Data set for Exercises 6.3 and 9.13


                                    Premarital Sex
                                   1                     2
    Religious Attendence       1        2           1          2  
         Birth control      1     2   1    2      1    2     1    2 
                      1     99   15   73  25       8   4     24  22 
 Political            2     73   20   87  37      20  13     50  60
 Views                3     51   19   51  36       6  12     33  88

9. Data set for Exercise 6.7


                   <35     35-44     >44        <35    35-44    >44

Region           M    F    M   F    M    F     M   F   M   F   M   F

Northeast
 Satisfied      288   60  224  35  337   70    38  19  32  22  21  15 
 Not satisfied  177   57  166  19  172   30    33  35  11  20   8  10 

Mid-Atlantic 
 Satisfied       90   19   96  12  124   17    18  13   7   0   9   1 
 Not satisfied   45   12   42   5   39    2     6   7   2   3   2   1 

Southern 
 Satisfied      226   88  189  44  156   70    45  47  18  13  11   9 
 Not satisfied  128   57  117  34   73   25    31  35   3   7   2   2  

Midwest  
 Satisfied      285  110  225  53  324   60    40  66  19  25  22  11  
 Not satisfied  179   93  141  24  140   47    25  56  11  19   2  12  

Northwest 
 Satisfied      270  176  215  80  269  110    36  25   9  11  16   4 
 Not satisfied  180  151  108  40  136   40    20  16   7   5   3   5 

Southwest 
 Satisfied      252   97  162  47  199   62    69  45  14   8  14   2 
 Not satisfied  126   61   72  27   93   24    27  36   7   4   5   0 

Pacific 
 Satisfied      119   62   66  20   67   25    45  22  15  10   8   6 
 Not satisfied   58   33   20  10   21   10    16  15  10   8   6   2 

10. Data on surgery and sore throats in Table 6.15, for Exercise 6.8


D  T  Y  
45 0 0 
15 0 0 
40 0 1 
83 1 1 
90 1 1 
25 1 1
35 0 1
65 0 1
95 0 1
35 0 1
75 0 1
45 1 1
50 1 0
75 1 1
30 0 0
25 0 1
20 1 0
60 1 1
70 1 1
30 0 1
60 0 1
61 0 0
65 0 1
15 1 0
20 1 0
45 0 1
15 1 0
25 0 1
15 1 0
30 0 1
40 0 1
15 1 0
135 1 1
20 1 0
40 1 0

11. Data on incontinence study, for Exercise 6.20


y x1 x2 x3
0  -1.9  -5.3  -43      
0  -0.1  -5.2  -32      
0  ~0.8  -3.0  -12      
0  ~0.9   3.4   ~1      
1  -5.6 -13.1   -1      
1  -2.4   1.8   -9      
1  -2.0  -5.7   -7      
1  -0.6  -2.4   -7      
1  -0.1 -10.2   -5      
1  ~0.4 -17.2   -9      
1  ~1.1  -4.5  -15                         
0  -1.5   3.9  -15
0   0.5  27.5    8
0   0.8  -1.6   -2
0   2.3  23.4   14
1  -5.3 -19.8  -33
1  -2.3  -7.4    4
1  -1.7  -3.9   13
1  -0.5 -14.5  -12
1  -0.1  -9.9  -11
1   0.7 -10.7  -10

12. Data set for Exercise 6.28


                                         Occupational aspirations
                         Socioeconomic
Gender  Residence   IQ     status
                                               High  Low

Male     Rural     High      High              117  47
                              Low               54  87 
                    Low      High               29  78 
                              Low               31  262 
         Small     High      High              350  80 
           urban              Low               70  85 
                    Low      High               71  120 
                              Low               33  265 
         Large     High      High              151   31 
           urban              Low               27   23 
                    Low      High               30   27 
                              Low               12   52 

Female   Rural     High      High              102   69 
                              Low               52  119 
                    Low      High               32   73 
                              Low               28  349 
         Small     High      High              338   96 
           urban              Low               44   99 
                    Low      High               76  107 
                              Low               22  344 
         Large     High      High              148   35 
           urban              Low               17   39 
                    Low      High               21   47 
                              Low                6  116 

13. Alligator food choice data set of Table 8.1


lake gender size food count
1 1 1 1 7   
1 1 1 2 1   
1 1 1 3 0   
1 1 1 4 0   
1 1 1 5 5
1 1 2 1 4   
1 1 2 2 0   
1 1 2 3 0   
1 1 2 4 1   
1 1 2 5 2        
1 2 1 1 16  
1 2 1 2 3   
1 2 1 3 2   
1 2 1 4 2   
1 2 1 5 3
1 2 2 1 3   
1 2 2 2 0   
1 2 2 3 1   
1 2 2 4 2   
1 2 2 5 3
2 1 1 1 2   
2 1 1 2 2   
2 1 1 3 0   
2 1 1 4 0   
2 1 1 5 1  
2 1 2 1 13  
2 1 2 2 7   
2 1 2 3 6   
2 1 2 4 0   
2 1 2 5 0  
2 2 1 1 3   
2 2 1 2 9   
2 2 1 3 1   
2 2 1 4 0   
2 2 1 5 2  
2 2 2 1 0   
2 2 2 2 1   
2 2 2 3 0   
2 2 2 4 1   
2 2 2 5 0 
3 1 1 1 3   
3 1 1 2 7   
3 1 1 3 1   
3 1 1 4 0   
3 1 1 5 1
3 1 2 1 8   
3 1 2 2 6   
3 1 2 3 6   
3 1 2 4 3   
3 1 2 5 5  
3 2 1 1 2   
3 2 1 2 4   
3 2 1 3 1   
3 2 1 4 1   
3 2 1 5 4          
3 2 2 1 0   
3 2 2 2 1   
3 2 2 3 0   
3 2 2 4 0   
3 2 2 5 0  
4 1 1 1 13  
4 1 1 2 10  
4 1 1 3 0   
4 1 1 4 2   
4 1 1 5 2  
4 1 2 1 9   
4 1 2 2 0   
4 1 2 3 0   
4 1 2 4 1   
4 1 2 5 2  
4 2 1 1 3   
4 2 1 2 9   
4 2 1 3 1   
4 2 1 4 0   
4 2 1 5 1
4 2 2 1 8   
4 2 2 2 1   
4 2 2 3 0   
4 2 2 4 0   
4 2 2 5 1           

14. Full data set for Table 8.5 on happiness, traumatic events, and race


  race trauma happy * race is 0=white, 1=black
    0 0 1 
    0 0 1 
    0 0 1 
    0 0 1 
    0 0 1 
    0 0 1 
    0 0 1 
    0 0 2 
    0 0 2 
    0 0 2 
    0 0 2
    0 0 2
    0 0 2
    0 0 2
    0 0 2 
    0 0 2 
    0 0 2 
    0 0 2
    0 0 2
    0 0 2
    0 0 2
    0 0 2
    0 0 3
    0 1 1
    0 1 1
    0 1 1
    0 1 1 
    0 1 1
    0 1 1
    0 1 1
    0 1 1 
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 2
    0 1 3
    0 2 1
    0 2 1
    0 2 1
    0 2 1
    0 2 1
    0 2 2
    0 2 2
    0 2 2
    0 2 2         
    0 2 2
    0 2 2
    0 2 2
    0 2 2         
    0 2 2
    0 2 2
    0 2 2
    0 2 2         
    0 2 2
    0 2 2
    0 2 2
    0 2 2         
    0 2 3
    0 3 1
    0 3 2
    0 3 2
    0 3 2 
    0 3 2
    0 3 2
    0 3 2 
    0 3 2
    0 3 2
    0 3 2 
    0 3 3
    0 4 1
    0 4 2
    0 4 2
    0 4 2
    0 4 2
    0 5 3
    0 5 3
    1 0 2
    1 0 3
    1 1 2
    1 1 2
    1 1 2
    1 1 3
    1 2 2
    1 2 2
    1 2 2
    1 2 3
    1 3 2
    1 3 2
    1 3 3

15. Data for Exercise 8.18 on dumping severity


            Hospital 1     Hosptial 2     Hospital 3    Hospital 4
 Operation   N   S   M      N   S   M      N   S   M     N   S  M 
   A        23    7   2    18    6   1     8   6   3    12   9   1 
   B        23   10   5    18    6   2    12   4   4    15   3   2 
   C        20   13   5    13   13   2    11   6   2    14   8   3 
   D        24   10   6     9   15   2     7   7   4    13   6   4 

16. Data for Exercise 8.28 on satisfaction with housing


Housing                    Low contact: Satisfaction    High contact: Satisfaction   
                             Low   Medium   High          Low   Medium   High 
                 Influence

Tower blocks         Low     21      21       28           14     19     37 
                  Medium     34      22       36           17     23     40 
                    High     10      11       36            3      5     23 
                                                                 
Apartments           Low     61      23       17           78     46     43 
                  Medium     43      35       40           48     45     86 
                    High     26      18       54           15     25     62 
                                                                 
Atrium houses        Low     13       9       10           20     23     20 
                  Medium      8       8       12           10     22     24 
                    High      6       7        9            7     10     21 
                                                                 
Terraced houses      Low     18       6        7           57     23     13 
                  Medium     15      13       13           31     21     13 
                    High      7       5       11            5      6     13 

17. High school student survey data set of Table 9.3


 a c m count 
1 1 1 911  
1 1 2 538   
1 2 1 44   
1 2 2 456
2 1 1   3   
2 1 2  43   
2 2 1  2   
2 2 2 279

18. Government spending data set for Exercise 9.5


e h c l count
1 1 1 1 62 
1 1 2 1 90 
1 1 3 1 74 
1 2 1 1 11 
1 2 2 1 22 
1 2 3 1 19 
1 3 1 1 2  
1 3 2 1 2  
1 3 3 1 1  
2 1 1 1 11 
2 1 2 1 21 
2 1 3 1 20 
2 2 1 1 1  
2 2 2 1 6  
2 2 3 1 6  
2 3 1 1 1  
2 3 2 1 2  
2 3 3 1 4  
3 1 1 1 3  
3 1 2 1 2  
3 1 3 1 9  
3 2 1 1 1  
3 2 2 1 2  
3 2 3 1 4  
3 3 1 1 1  
3 3 2 1 0  
3 3 3 1 1  
1 1 1 2 17     
 1 1 2 2 42 
 1 1 3 2 31 
 1 2 1 2 7  
 1 2 2 2 18 
 1 2 3 2 14 
1 3 1 2 3      
1 3 2 2 0      
1 3 3 2 3      
 2 1 1 2 3  
 2 1 2 2 13 
 2 1 3 2 8  
2 2 1 2 4   
2 2 2 2 9      
2 2 3 2 5      
2 3 1 2 0      
2 3 2 2 1      
2 3 3 2 3      
3 1 1 2 0      
3 1 2 2 1      
3 1 3 2 2      
3 2 1 2 0      
3 2 2 2 1      
3 2 3 2 2      
3 3 1 2 0      
3 3 2 2 0      
3 3 3 2 2   
1 1 1 3 5	
  1 1 2 3 3  
  1 1 3 3 11 
 1 2 1 3 0   
  1 2 2 3 1  
  1 2 3 3 3  
1 3 1 3 1	
1 3 2 3 1	
1 3 3 3 1	
 2 1 1 3 0   
  2 1 2 3 2  
 2 1 3 3 3   
2 2 1 3 0    
2 2 2 3 0	
2 2 3 3 2	
2 3 1 3 1	
2 3 2 3 1	
2 3 3 3 1	
3 1 1 3 0	
3 1 2 3 0	
3 1 3 3 1	
3 2 1 3 0	
3 2 2 3 0	
3 2 3 3 0	
3 3 1 3 0	
3 3 2 3 0	
3 3 3 3 3    

19. Alcohol, cigarette, and marijuana use, by gender and race, as in Table 10.1


a c m r g count 
1 1 1 1 1 405      
1 1 1 2 1  23      
1 2 1 1 1  13      
1 2 1 2 1   2      
2 1 1 1 1   1      
2 1 1 2 1   0      
2 2 1 1 1   1      
2 2 1 2 1   0      
1 1 2 1 1 268
1 1 2 2 1  23
1 2 2 1 1 218
1 2 2 2 1  19
2 1 2 1 1  17
2 1 2 2 1   1
2 2 2 1 1 117
2 2 2 2 1  12
1 1 1 1 2 453
1 1 1 2 2  30
1 2 1 1 2  28
1 2 1 2 2   1
2 1 1 1 2   1
2 1 1 2 2   1
2 2 1 1 2   1
2 2 1 2 2   0
1 1 2 1 2 228
1 1 2 2 2  19
1 2 2 1 2 201
1 2 2 2 2  18
2 1 2 1 2  17
2 1 2 2 2   8
2 2 2 1 2 133
2 2 2 2 2  17

20. Opinions about birth control and premarital sex data set of Table 10.3


premar birth count  
1 4  38   
1 3  60   
1 2  68   
1 1  81
2 4  14    
2 3  29   
2 2  26   
2 1  24
3 4  42   
3 3  74   
3 2  41   
3 1  18
4 4 157   
4 3 161   
4 2  57  
4 1  36

21. Migration data set of Table 11.5


 row column count
 ne ne 266  
  ne mw  15 
  ne  s  61 
  ne  w  28 
  mw ne  10 
  mw mw 414 
  mw  s  50 
  mw  w  40 
   s ne   8 
   s mw  22 
   s  s 578 
   s  w  22 
   w ne   7 
   w mw   6 
   w  s  27 
   w  w 301 

22. Depression data set of Table 12.1 in case form


case severity treat time outcome * outcome=1 is normal
  1  0  0  0  1
  1  0  0  1  1
  1  0  0  2  1
  2  0  0  0  1
  2  0  0  1  1
  2  0  0  2  1
  3  0  0  0  1
  3  0  0  1  1
  3  0  0  2  1
  4  0  0  0  1
  4  0  0  1  1
  4  0  0  2  1
  5  0  0  0  1
  5  0  0  1  1
  5  0  0  2  1
  6  0  0  0  1
  6  0  0  1  1
  6  0  0  2  1
  7  0  0  0  1
  7  0  0  1  1
  7  0  0  2  1
  8  0  0  0  1
  8  0  0  1  1
  8  0  0  2  1
  9  0  0  0  1
  9  0  0  1  1
  9  0  0  2  1
 10  0  0  0  1
 10  0  0  1  1
 10  0  0  2  1
 11  0  0  0  1
 11  0  0  1  1
 11  0  0  2  1
 12  0  0  0  1
 12  0  0  1  1
 12  0  0  2  1
 13  0  0  0  1
 13  0  0  1  1
 13  0  0  2  1
 14  0  0  0  1
 14  0  0  1  1
 14  0  0  2  1
 15  0  0  0  1
 15  0  0  1  1
 15  0  0  2  1
 16  0  0  0  1
 16  0  0  1  1
 16  0  0  2  1
 17  0  0  0  1
 17  0  0  1  1
 17  0  0  2  0
 18  0  0  0  1
 18  0  0  1  1
 18  0  0  2  0
 19  0  0  0  1
 19  0  0  1  1
 19  0  0  2  0
 20  0  0  0  1
 20  0  0  1  1
 20  0  0  2  0
 21  0  0  0  1
 21  0  0  1  1
 21  0  0  2  0
 22  0  0  0  1
 22  0  0  1  1
 22  0  0  2  0
 23  0  0  0  1
 23  0  0  1  1
 23  0  0  2  0
 24  0  0  0  1
 24  0  0  1  1
 24  0  0  2  0
 25  0  0  0  1
 25  0  0  1  1
 25  0  0  2  0
 26  0  0  0  1
 26  0  0  1  1
 26  0  0  2  0
 27  0  0  0  1
 27  0  0  1  1
 27  0  0  2  0
 28  0  0  0  1
 28  0  0  1  1
 28  0  0  2  0
 29  0  0  0  1
 29  0  0  1  1
 29  0  0  2  0
 30  0  0  0  1
 30  0  0  1  0
 30  0  0  2  1
 31  0  0  0  1
 31  0  0  1  0
 31  0  0  2  1
 32  0  0  0  1
 32  0  0  1  0
 32  0  0  2  1
 33  0  0  0  1
 33  0  0  1  0
 33  0  0  2  1
 34  0  0  0  1
 34  0  0  1  0
 34  0  0  2  1
 35  0  0  0  1
 35  0  0  1  0
 35  0  0  2  1
 36  0  0  0  1
 36  0  0  1  0
 36  0  0  2  1
 37  0  0  0  1
 37  0  0  1  0
 37  0  0  2  1
 38  0  0  0  1
 38  0  0  1  0
 38  0  0  2  1
 39  0  0  0  1
 39  0  0  1  0
 39  0  0  2  0
 40  0  0  0  1
 40  0  0  1  0
 40  0  0  2  0
 41  0  0  0  1
 41  0  0  1  0
 41  0  0  2  0
 42  0  0  0  0
 42  0  0  1  1
 42  0  0  2  1
 43  0  0  0  0
 43  0  0  1  1
 43  0  0  2  1
 44  0  0  0  0
 44  0  0  1  1
 44  0  0  2  1
 45  0  0  0  0
 45  0  0  1  1
 45  0  0  2  1
 46  0  0  0  0
 46  0  0  1  1
 46  0  0  2  1
 47  0  0  0  0
 47  0  0  1  1
 47  0  0  2  1
 48  0  0  0  0
 48  0  0  1  1
 48  0  0  2  1
 49  0  0  0  0
 49  0  0  1  1
 49  0  0  2  1
 50  0  0  0  0
 50  0  0  1  1
 50  0  0  2  1
 51  0  0  0  0
 51  0  0  1  1
 51  0  0  2  1
 52  0  0  0  0
 52  0  0  1  1
 52  0  0  2  1
 53  0  0  0  0
 53  0  0  1  1
 53  0  0  2  1
 54  0  0  0  0
 54  0  0  1  1
 54  0  0  2  1
 55  0  0  0  0
 55  0  0  1  1
 55  0  0  2  1
 56  0  0  0  0
 56  0  0  1  1
 56  0  0  2  0
 57  0  0  0  0
 57  0  0  1  1
 57  0  0  2  0
 58  0  0  0  0
 58  0  0  1  1
 58  0  0  2  0
 59  0  0  0  0
 59  0  0  1  1
 59  0  0  2  0
 60  0  0  0  0
 60  0  0  1  0
 60  0  0  2  1
 61  0  0  0  0
 61  0  0  1  0
 61  0  0  2  1
 62  0  0  0  0
 62  0  0  1  0
 62  0  0  2  1
 63  0  0  0  0
 63  0  0  1  0
 63  0  0  2  1
 64  0  0  0  0
 64  0  0  1  0
 64  0  0  2  1
 65  0  0  0  0
 65  0  0  1  0
 65  0  0  2  1
 66  0  0  0  0
 66  0  0  1  0
 66  0  0  2  1
 67  0  0  0  0
 67  0  0  1  0
 67  0  0  2  1
 68  0  0  0  0
 68  0  0  1  0
 68  0  0  2  1
 69  0  0  0  0
 69  0  0  1  0
 69  0  0  2  1
 70  0  0  0  0
 70  0  0  1  0
 70  0  0  2  1
 71  0  0  0  0
 71  0  0  1  0
 71  0  0  2  1
 72  0  0  0  0
 72  0  0  1  0
 72  0  0  2  1
 73  0  0  0  0
 73  0  0  1  0
 73  0  0  2  1
 74  0  0  0  0
 74  0  0  1  0
 74  0  0  2  1
 75  0  0  0  0
 75  0  0  1  0
 75  0  0  2  0
336  0  0  0  0
336  0  0  1  0
336  0  0  2  0
337  0  0  0  0
337  0  0  1  0
337  0  0  2  0
338  0  0  0  0
338  0  0  1  0
338  0  0  2  0
339  0  0  0  0
339  0  0  1  0
339  0  0  2  0
340  0  0  0  0
340  0  0  1  0
340  0  0  2  0

 76  0  1  0  1
 76  0  1  1  1
 76  0  1  2  1
 77  0  1  0  1
 77  0  1  1  1
 77  0  1  2  1
 78  0  1  0  1
 78  0  1  1  1
 78  0  1  2  1
 79  0  1  0  1
 79  0  1  1  1
 79  0  1  2  1
 80  0  1  0  1
 80  0  1  1  1
 80  0  1  2  1
 81  0  1  0  1
 81  0  1  1  1
 81  0  1  2  1
 82  0  1  0  1
 82  0  1  1  1
 82  0  1  2  1
 83  0  1  0  1
 83  0  1  1  1
 83  0  1  2  1
 84  0  1  0  1
 84  0  1  1  1
 84  0  1  2  1
 85  0  1  0  1
 85  0  1  1  1
 85  0  1  2  1
 86  0  1  0  1
 86  0  1  1  1
 86  0  1  2  1
 87  0  1  0  1
 87  0  1  1  1
 87  0  1  2  1
 88  0  1  0  1
 88  0  1  1  1
 88  0  1  2  1
 89  0  1  0  1
 89  0  1  1  1
 89  0  1  2  1
 90  0  1  0  1
 90  0  1  1  1
 90  0  1  2  1
 91  0  1  0  1
 91  0  1  1  1
 91  0  1  2  1
 92  0  1  0  1
 92  0  1  1  1
 92  0  1  2  1
 93  0  1  0  1
 93  0  1  1  1
 93  0  1  2  1
 94  0  1  0  1
 94  0  1  1  1
 94  0  1  2  1
 95  0  1  0  1
 95  0  1  1  1
 95  0  1  2  1
 96  0  1  0  1
 96  0  1  1  1
 96  0  1  2  1
 97  0  1  0  1
 97  0  1  1  1
 97  0  1  2  1
 98  0  1  0  1
 98  0  1  1  1
 98  0  1  2  1
 99  0  1  0  1
 99  0  1  1  1
 99  0  1  2  1
100  0  1  0  1
100  0  1  1  1
100  0  1  2  1
101  0  1  0  1
101  0  1  1  1
101  0  1  2  1
102  0  1  0  1
102  0  1  1  1
102  0  1  2  1
103  0  1  0  1
103  0  1  1  1
103  0  1  2  1
104  0  1  0  1
104  0  1  1  1
104  0  1  2  1
105  0  1  0  1
105  0  1  1  1
105  0  1  2  1
106  0  1  0  1
106  0  1  1  1
106  0  1  2  1
107  0  1  0  1
107  0  1  1  0
107  0  1  2  1
108  0  1  0  1
108  0  1  1  0
108  0  1  2  1
109  0  1  0  1
109  0  1  1  0
109  0  1  2  1
110  0  1  0  1
110  0  1  1  0
110  0  1  2  1
111  0  1  0  1
111  0  1  1  0
111  0  1  2  1
112  0  1  0  1
112  0  1  1  0
112  0  1  2  1
113  0  1  0  0
113  0  1  1  1
113  0  1  2  1
114  0  1  0  0
114  0  1  1  1
114  0  1  2  1
115  0  1  0  0
115  0  1  1  1
115  0  1  2  1
116  0  1  0  0
116  0  1  1  1
116  0  1  2  1
117  0  1  0  0
117  0  1  1  1
117  0  1  2  1
118  0  1  0  0
118  0  1  1  1
118  0  1  2  1
119  0  1  0  0
119  0  1  1  1
119  0  1  2  1
120  0  1  0  0
120  0  1  1  1
120  0  1  2  1
121  0  1  0  0
121  0  1  1  1
121  0  1  2  1
122  0  1  0  0
122  0  1  1  1
122  0  1  2  1
123  0  1  0  0
123  0  1  1  1
123  0  1  2  1
124  0  1  0  0
124  0  1  1  1
124  0  1  2  1
125  0  1  0  0
125  0  1  1  1
125  0  1  2  1
126  0  1  0  0
126  0  1  1  1
126  0  1  2  1
127  0  1  0  0
127  0  1  1  1
127  0  1  2  1
128  0  1  0  0
128  0  1  1  1
128  0  1  2  1
129  0  1  0  0
129  0  1  1  1
129  0  1  2  1
130  0  1  0  0
130  0  1  1  1
130  0  1  2  1
131  0  1  0  0
131  0  1  1  1
131  0  1  2  1
132  0  1  0  0
132  0  1  1  1
132  0  1  2  1
133  0  1  0  0
133  0  1  1  1
133  0  1  2  1
134  0  1  0  0
134  0  1  1  1
134  0  1  2  1
135  0  1  0  0
135  0  1  1  1
135  0  1  2  0
136  0  1  0  0
136  0  1  1  1
136  0  1  2  0
137  0  1  0  0
137  0  1  1  0
137  0  1  2  1
138  0  1  0  0
138  0  1  1  0
138  0  1  2  1
139  0  1  0  0
139  0  1  1  0
139  0  1  2  1
140  0  1  0  0
140  0  1  1  0
140  0  1  2  1
141  0  1  0  0
141  0  1  1  0
141  0  1  2  1
142  0  1  0  0
142  0  1  1  0
142  0  1  2  1
143  0  1  0  0
143  0  1  1  0
143  0  1  2  1
144  0  1  0  0
144  0  1  1  0
144  0  1  2  1
145  0  1  0  0
145  0  1  1  0
145  0  1  2  1

146  1  0  0  1
146  1  0  1  1
146  1  0  2  1
147  1  0  0  1
147  1  0  1  1
147  1  0  2  1
148  1  0  0  1
148  1  0  1  1
148  1  0  2  0
149  1  0  0  1
149  1  0  1  1
149  1  0  2  0
150  1  0  0  1
150  1  0  1  0
150  1  0  2  1
151  1  0  0  1
151  1  0  1  0
151  1  0  2  1
152  1  0  0  1
152  1  0  1  0
152  1  0  2  1
153  1  0  0  1
153  1  0  1  0
153  1  0  2  1
154  1  0  0  1
154  1  0  1  0
154  1  0  2  1
155  1  0  0  1
155  1  0  1  0
155  1  0  2  1
156  1  0  0  1
156  1  0  1  0
156  1  0  2  1
157  1  0  0  1
157  1  0  1  0
157  1  0  2  1
158  1  0  0  1
158  1  0  1  0
158  1  0  2  0
159  1  0  0  1
159  1  0  1  0
159  1  0  2  0
160  1  0  0  1
160  1  0  1  0
160  1  0  2  0
161  1  0  0  1
161  1  0  1  0
161  1  0  2  0
162  1  0  0  1
162  1  0  1  0
162  1  0  2  0
163  1  0  0  1
163  1  0  1  0
163  1  0  2  0
164  1  0  0  1
164  1  0  1  0
164  1  0  2  0
165  1  0  0  1
165  1  0  1  0
165  1  0  2  0
166  1  0  0  1
166  1  0  1  0
166  1  0  2  0
167  1  0  0  0
167  1  0  1  1
167  1  0  2  1
168  1  0  0  0
168  1  0  1  1
168  1  0  2  1
169  1  0  0  0
169  1  0  1  1
169  1  0  2  1
170  1  0  0  0
170  1  0  1  1
170  1  0  2  1
171  1  0  0  0
171  1  0  1  1
171  1  0  2  1
172  1  0  0  0
172  1  0  1  1
172  1  0  2  1
173  1  0  0  0
173  1  0  1  1
173  1  0  2  1
174  1  0  0  0
174  1  0  1  1
174  1  0  2  1
175  1  0  0  0
175  1  0  1  1
175  1  0  2  1
176  1  0  0  0
176  1  0  1  1
176  1  0  2  0
177  1  0  0  0
177  1  0  1  1
177  1  0  2  0
178  1  0  0  0
178  1  0  1  1
178  1  0  2  0
179  1  0  0  0
179  1  0  1  1
179  1  0  2  0
180  1  0  0  0
180  1  0  1  1
180  1  0  2  0
181  1  0  0  0
181  1  0  1  1
181  1  0  2  0
182  1  0  0  0
182  1  0  1  1
182  1  0  2  0
183  1  0  0  0
183  1  0  1  1
183  1  0  2  0
184  1  0  0  0
184  1  0  1  1
184  1  0  2  0
185  1  0  0  0
185  1  0  1  1
185  1  0  2  0
186  1  0  0  0
186  1  0  1  1
186  1  0  2  0
187  1  0  0  0
187  1  0  1  1
187  1  0  2  0
188  1  0  0  0
188  1  0  1  1
188  1  0  2  0
189  1  0  0  0
189  1  0  1  1
189  1  0  2  0
190  1  0  0  0
190  1  0  1  1
190  1  0  2  0
191  1  0  0  0
191  1  0  1  0
191  1  0  2  1
192  1  0  0  0
192  1  0  1  0
192  1  0  2  1
193  1  0  0  0
193  1  0  1  0
193  1  0  2  1
194  1  0  0  0
194  1  0  1  0
194  1  0  2  1
195  1  0  0  0
195  1  0  1  0
195  1  0  2  1
196  1  0  0  0
196  1  0  1  0
196  1  0  2  1
197  1  0  0  0
197  1  0  1  0
197  1  0  2  1
198  1  0  0  0
198  1  0  1  0
198  1  0  2  1
199  1  0  0  0
199  1  0  1  0
199  1  0  2  1
200  1  0  0  0
200  1  0  1  0
200  1  0  2  1
201  1  0  0  0
201  1  0  1  0
201  1  0  2  1
202  1  0  0  0
202  1  0  1  0
202  1  0  2  1
203  1  0  0  0
203  1  0  1  0
203  1  0  2  1
204  1  0  0  0
204  1  0  1  0
204  1  0  2  1
205  1  0  0  0
205  1  0  1  0
205  1  0  2  1
206  1  0  0  0
206  1  0  1  0
206  1  0  2  1
207  1  0  0  0
207  1  0  1  0
207  1  0  2  1
208  1  0  0  0
208  1  0  1  0
208  1  0  2  1
209  1  0  0  0
209  1  0  1  0
209  1  0  2  1
210  1  0  0  0
210  1  0  1  0
210  1  0  2  1
211  1  0  0  0
211  1  0  1  0
211  1  0  2  1
212  1  0  0  0
212  1  0  1  0
212  1  0  2  1
213  1  0  0  0
213  1  0  1  0
213  1  0  2  1
214  1  0  0  0
214  1  0  1  0
214  1  0  2  1
215  1  0  0  0
215  1  0  1  0
215  1  0  2  1
216  1  0  0  0
216  1  0  1  0
216  1  0  2  1
217  1  0  0  0
217  1  0  1  0
217  1  0  2  1
218  1  0  0  0
218  1  0  1  0
218  1  0  2  0
219  1  0  0  0
219  1  0  1  0
219  1  0  2  0
220  1  0  0  0
220  1  0  1  0
220  1  0  2  0
221  1  0  0  0
221  1  0  1  0
221  1  0  2  0
222  1  0  0  0
222  1  0  1  0
222  1  0  2  0
223  1  0  0  0
223  1  0  1  0
223  1  0  2  0
224  1  0  0  0
224  1  0  1  0
224  1  0  2  0
225  1  0  0  0
225  1  0  1  0
225  1  0  2  0
226  1  0  0  0
226  1  0  1  0
226  1  0  2  0
227  1  0  0  0
227  1  0  1  0
227  1  0  2  0
228  1  0  0  0
228  1  0  1  0
228  1  0  2  0
229  1  0  0  0
229  1  0  1  0
229  1  0  2  0
230  1  0  0  0
230  1  0  1  0
230  1  0  2  0
231  1  0  0  0
231  1  0  1  0
231  1  0  2  0
232  1  0  0  0
232  1  0  1  0
232  1  0  2  0
233  1  0  0  0
233  1  0  1  0
233  1  0  2  0
234  1  0  0  0
234  1  0  1  0
234  1  0  2  0
235  1  0  0  0
235  1  0  1  0
235  1  0  2  0
236  1  0  0  0
236  1  0  1  0
236  1  0  2  0
237  1  0  0  0
237  1  0  1  0
237  1  0  2  0
238  1  0  0  0
238  1  0  1  0
238  1  0  2  0
239  1  0  0  0
239  1  0  1  0
239  1  0  2  0
240  1  0  0  0
240  1  0  1  0
240  1  0  2  0
241  1  0  0  0
241  1  0  1  0
241  1  0  2  0
242  1  0  0  0
242  1  0  1  0
242  1  0  2  0
243  1  0  0  0
243  1  0  1  0
243  1  0  2  0
244  1  0  0  0
244  1  0  1  0
244  1  0  2  0
245  1  0  0  0
245  1  0  1  0
245  1  0  2  0

246  1  1  0  1
246  1  1  1  1
246  1  1  2  1
247  1  1  0  1
247  1  1  1  1
247  1  1  2  1
248  1  1  0  1
248  1  1  1  1
248  1  1  2  1
249  1  1  0  1
249  1  1  1  1
249  1  1  2  1
250  1  1  0  1
250  1  1  1  1
250  1  1  2  1
251  1  1  0  1
251  1  1  1  1
251  1  1  2  1
252  1  1  0  1
252  1  1  1  1
252  1  1  2  1
253  1  1  0  1
253  1  1  1  1
253  1  1  2  0 
254  1  1  0  1
254  1  1  1  1
254  1  1  2  0 
255  1  1  0  1
255  1  1  1  0
255  1  1  2  1
256  1  1  0  1
256  1  1  1  0
256  1  1  2  1
257  1  1  0  1
257  1  1  1  0
257  1  1  2  1
258  1  1  0  1
258  1  1  1  0
258  1  1  2  1
259  1  1  0  1
259  1  1  1  0
259  1  1  2  1
260  1  1  0  1
260  1  1  1  0
260  1  1  2  0
261  1  1  0  1
261  1  1  1  0
261  1  1  2  0
262  1  1  0  0
262  1  1  1  1
262  1  1  2  1
263  1  1  0  0
263  1  1  1  1
263  1  1  2  1
264  1  1  0  0
264  1  1  1  1
264  1  1  2  1
265  1  1  0  0
265  1  1  1  1
265  1  1  2  1
266  1  1  0  0
266  1  1  1  1
266  1  1  2  1
267  1  1  0  0
267  1  1  1  1
267  1  1  2  1
268  1  1  0  0
268  1  1  1  1
268  1  1  2  1
269  1  1  0  0
269  1  1  1  1
269  1  1  2  1
270  1  1  0  0
270  1  1  1  1
270  1  1  2  1
271  1  1  0  0
271  1  1  1  1
271  1  1  2  1
272  1  1  0  0
272  1  1  1  1
272  1  1  2  1
273  1  1  0  0
273  1  1  1  1
273  1  1  2  1
274  1  1  0  0
274  1  1  1  1
274  1  1  2  1
275  1  1  0  0
275  1  1  1  1
275  1  1  2  1
276  1  1  0  0
276  1  1  1  1
276  1  1  2  1
277  1  1  0  0
277  1  1  1  1
277  1  1  2  1
278  1  1  0  0
278  1  1  1  1
278  1  1  2  1
279  1  1  0  0
279  1  1  1  1
279  1  1  2  1
280  1  1  0  0
280  1  1  1  1
280  1  1  2  1
281  1  1  0  0
281  1  1  1  1
281  1  1  2  1
282  1  1  0  0
282  1  1  1  1
282  1  1  2  1
283  1  1  0  0
283  1  1  1  1
283  1  1  2  1
284  1  1  0  0
284  1  1  1  1
284  1  1  2  1
285  1  1  0  0
285  1  1  1  1
285  1  1  2  1
286  1  1  0  0
286  1  1  1  1
286  1  1  2  1
287  1  1  0  0
287  1  1  1  1
287  1  1  2  1
288  1  1  0  0
288  1  1  1  1
288  1  1  2  1
289  1  1  0  0
289  1  1  1  1
289  1  1  2  1
290  1  1  0  0
290  1  1  1  1
290  1  1  2  1
291  1  1  0  0
291  1  1  1  1
291  1  1  2  1
292  1  1  0  0
292  1  1  1  1
292  1  1  2  1
293  1  1  0  0
293  1  1  1  1 
293  1  1  2  0
294  1  1  0  0
294  1  1  1  1 
294  1  1  2  0
295  1  1  0  0
295  1  1  1  1 
295  1  1  2  0
296  1  1  0  0
296  1  1  1  1 
296  1  1  2  0
297  1  1  0  0
297  1  1  1  1 
297  1  1  2  0
298  1  1  0  0
298  1  1  1  0
298  1  1  2  1
299  1  1  0  0
299  1  1  1  0
299  1  1  2  1
300  1  1  0  0
300  1  1  1  0
300  1  1  2  1
301  1  1  0  0
301  1  1  1  0
301  1  1  2  1
302  1  1  0  0
302  1  1  1  0
302  1  1  2  1
303  1  1  0  0
303  1  1  1  0
303  1  1  2  1
304  1  1  0  0
304  1  1  1  0
304  1  1  2  1
305  1  1  0  0
305  1  1  1  0
305  1  1  2  1
306  1  1  0  0
306  1  1  1  0
306  1  1  2  1
307  1  1  0  0
307  1  1  1  0
307  1  1  2  1
308  1  1  0  0
308  1  1  1  0
308  1  1  2  1
309  1  1  0  0
309  1  1  1  0
309  1  1  2  1
310  1  1  0  0
310  1  1  1  0
310  1  1  2  1
311  1  1  0  0
311  1  1  1  0
311  1  1  2  1
312  1  1  0  0
312  1  1  1  0
312  1  1  2  1
313  1  1  0  0
313  1  1  1  0
313  1  1  2  1
314  1  1  0  0
314  1  1  1  0
314  1  1  2  1
315  1  1  0  0
315  1  1  1  0
315  1  1  2  1
316  1  1  0  0
316  1  1  1  0
316  1  1  2  1
317  1  1  0  0
317  1  1  1  0
317  1  1  2  1
318  1  1  0  0
318  1  1  1  0
318  1  1  2  1
319  1  1  0  0
319  1  1  1  0
319  1  1  2  1
320  1  1  0  0
320  1  1  1  0
320  1  1  2  1
321  1  1  0  0
321  1  1  1  0
321  1  1  2  1
322  1  1  0  0
322  1  1  1  0
322  1  1  2  1
323  1  1  0  0
323  1  1  1  0
323  1  1  2  1
324  1  1  0  0
324  1  1  1  0
324  1  1  2  1
325  1  1  0  0
325  1  1  1  0
325  1  1  2  1
326  1  1  0  0
326  1  1  1  0
326  1  1  2  1
327  1  1  0  0
327  1  1  1  0
327  1  1  2  1
328  1  1  0  0
328  1  1  1  0
328  1  1  2  1
329  1  1  0  0
329  1  1  1  0
329  1  1  2  1
330  1  1  0  0
330  1  1  1  0
330  1  1  2  0
331  1  1  0  0
331  1  1  1  0
331  1  1  2  0
332  1  1  0  0
332  1  1  1  0
332  1  1  2  0
333  1  1  0  0
333  1  1  1  0
333  1  1  2  0
334  1  1  0  0
334  1  1  1  0
334  1  1  2  0
335  1  1  0  0
335  1  1  1  0
335  1  1  2  0

23. Insomnia data set of Table 12.3


 input case treat occasion outcome count;
datalines;                   
        1       1     0        1   7
        1       1     1        1   7
        2       1     0        1   7
        2       1     1        1   7
        3       1     0        1   7
        3       1     1        1   7
        4       1     0        1   7
        4       1     1        1   7
        5       1     0        1   7
        5       1     1        1   7
        6       1     0        1   7
        6       1     1        1   7
        7       1     0        1   7
        7       1     1        1   7
        8       1     0        1   4
        8       1     1        2   4
        9       1     0        1   4
        9       1     1        2   4
       10       1     0        1   4
       10       1     1        2   4
       11       1     0        1   4
       11       1     1        2   4
       12       1     0        1   1
       12       1     1        3   1
       13       1     0        2  11
       13       1     1        1  11
       14       1     0        2  11
       14       1     1        1  11
       15       1     0        2  11
       15       1     1        1  11
       16       1     0        2  11
       16       1     1        1  11
       17       1     0        2  11
       17       1     1        1  11
       18       1     0        2  11
       18       1     1        1  11
       19       1     0        2  11
       19       1     1        1  11
       20       1     0        2  11
       20       1     1        1  11
       21       1     0        2  11
       21       1     1        1  11
       22       1     0        2  11
       22       1     1        1  11
       23       1     0        2  11
       23       1     1        1  11
       24       1     0        2   5
       24       1     1        2   5
       25       1     0        2   5
       25       1     1        2   5
       26       1     0        2   5
       26       1     1        2   5
       27       1     0        2   5
       27       1     1        2   5
       28       1     0        2   5
       28       1     1        2   5
       29       1     0        2   2
       29       1     1        3   2
       30       1     0        2   2
       30       1     1        3   2
       31       1     0        2   2
       31       1     1        4   2
       32       1     0        2   2
       32       1     1        4   2
       33       1     0        3  13
       33       1     1        1  13
       34       1     0        3  13
       34       1     1        1  13
       35       1     0        3  13
       35       1     1        1  13
       36       1     0        3  13
       36       1     1        1  13
       37       1     0        3  13
       37       1     1        1  13
       38       1     0        3  13
       38       1     1        1  13
       39       1     0        3  13
       39       1     1        1  13
       40       1     0        3  13
       40       1     1        1  13
       41       1     0        3  13
       41       1     1        1  13
       42       1     0        3  13
       42       1     1        1  13
       43       1     0        3  13
       43       1     1        1  13
       44       1     0        3  13
       44       1     1        1  13
       45       1     0        3  13
       45       1     1        1  13
       46       1     0        3  23
       46       1     1        2  23
       47       1     0        3  23
       47       1     1        2  23
       48       1     0        3  23
       48       1     1        2  23
       49       1     0        3  23
       49       1     1        2  23
       50       1     0        3  23
       50       1     1        2  23
       51       1     0        3  23
       51       1     1        2  23
       52       1     0        3  23
       52       1     1        2  23
       53       1     0        3  23
       53       1     1        2  23
       54       1     0        3  23
       54       1     1        2  23
       55       1     0        3  23
       55       1     1        2  23
       56       1     0        3  23
       56       1     1        2  23
       57       1     0        3  23
       57       1     1        2  23
       58       1     0        3  23
       58       1     1        2  23
       59       1     0        3  23
       59       1     1        2  23
       60       1     0        3  23
       60       1     1        2  23
       61       1     0        3  23
       61       1     1        2  23
       62       1     0        3  23
       62       1     1        2  23
       63       1     0        3  23
       63       1     1        2  23
       64       1     0        3  23
       64       1     1        2  23
       65       1     0        3  23
       65       1     1        2  23
       66       1     0        3  23
       66       1     1        2  23
       67       1     0        3  23
       67       1     1        2  23
       68       1     0        3  23
       68       1     1        2  23
       69       1     0        3   3
       69       1     1        3   3
       70       1     0        3   3
       70       1     1        3   3
       71       1     0        3   3
       71       1     1        3   3
       72       1     0        3   1
       72       1     1        4   1
       73       1     0        4   9
       73       1     1        1   9
       74       1     0        4   9
       74       1     1        1   9
       75       1     0        4   9
       75       1     1        1   9
       76       1     0        4   9
       76       1     1        1   9
       77       1     0        4   9
       77       1     1        1   9
       78       1     0        4   9
       78       1     1        1   9
       79       1     0        4   9
       79       1     1        1   9
       80       1     0        4   9
       80       1     1        1   9
       81       1     0        4   9
       81       1     1        1   9
       82       1     0        4  17
       82       1     1        2  17
       83       1     0        4  17
       83       1     1        2  17
       84       1     0        4  17
       84       1     1        2  17
       85       1     0        4  17
       85       1     1        2  17
       86       1     0        4  17
       86       1     1        2  17
       87       1     0        4  17
       87       1     1        2  17
       88       1     0        4  17
       88       1     1        2  17
       89       1     0        4  17
       89       1     1        2  17
       90       1     0        4  17
       90       1     1        2  17
       91       1     0        4  17
       91       1     1        2  17
       92       1     0        4  17
       92       1     1        2  17
       93       1     0        4  17
       93       1     1        2  17
       94       1     0        4  17
       94       1     1        2  17
       95       1     0        4  17
       95       1     1        2  17
       96       1     0        4  17
       96       1     1        2  17
       97       1     0        4  17
       97       1     1        2  17
       98       1     0        4  17
       98       1     1        2  17
       99       1     0        4  13
       99       1     1        3  13
      100       1     0        4  13
      100       1     1        3  13
      101       1     0        4  13
      101       1     1        3  13
      102       1     0        4  13
      102       1     1        3  13
      103       1     0        4  13
      103       1     1        3  13
      104       1     0        4  13
      104       1     1        3  13
      105       1     0        4  13
      105       1     1        3  13
      106       1     0        4  13
      106       1     1        3  13
      107       1     0        4  13
      107       1     1        3  13
      108       1     0        4  13
      108       1     1        3  13
      109       1     0        4  13
      109       1     1        3  13
      110       1     0        4  13
      110       1     1        3  13
      111       1     0        4  13
      111       1     1        3  13
      112       1     0        4   8
      112       1     1        4   8
      113       1     0        4   8
      113       1     1        4   8
      114       1     0        4   8
      114       1     1        4   8
      115       1     0        4   8
      115       1     1        4   8
      116       1     0        4   8
      116       1     1        4   8
      117       1     0        4   8
      117       1     1        4   8
      118       1     0        4   8
      118       1     1        4   8
      119       1     0        4   8
      119       1     1        4   8
      120       0     0        1   7
      120       0     1        1   7
      121       0     0        1   7
      121       0     1        1   7
      122       0     0        1   7
      122       0     1        1   7
      123       0     0        1   7
      123       0     1        1   7
      124       0     0        1   7
      124       0     1        1   7
      125       0     0        1   7
      125       0     1        1   7
      126       0     0        1   7
      126       0     1        1   7
      128       0     0        1   4
      128       0     1        2   4
      129       0     0        1   4
      129       0     1        2   4
      130       0     0        1   4
      130       0     1        2   4
      131       0     0        1   4
      131       0     1        2   4
      132       0     0        1   2
      132       0     1        3   2
      133       0     0        1   2
      133       0     1        3   2
      134       0     0        1   1
      134       0     1        4   1
      135       0     0        2  14
      135       0     1        1  14
      136       0     0        2  14
      136       0     1        1  14
      137       0     0        2  14
      137       0     1        1  14
      138       0     0        2  14
      138       0     1        1  14
      139       0     0        2  14
      139       0     1        1  14
      140       0     0        2  14
      140       0     1        1  14
      141       0     0        2  14
      141       0     1        1  14
      142       0     0        2  14
      142       0     1        1  14
      143       0     0        2  14
      143       0     1        1  14
      144       0     0        2  14
      144       0     1        1  14
      145       0     0        2  14
      145       0     1        1  14
      146       0     0        2  14
      146       0     1        1  14
      147       0     0        2  14
      147       0     1        1  14
      148       0     0        2  14
      148       0     1        1  14
      149       0     0        2   5
      149       0     1        2   5
      150       0     0        2   5
      150       0     1        2   5
      151       0     0        2   5
      151       0     1        2   5
      152       0     0        2   5
      152       0     1        2   5
      153       0     0        2   5
      153       0     1        2   5
      154       0     0        2   1
      154       0     1        3   1
      155       0     0        3   6
      155       0     1        1   6
      156       0     0        3   6
      156       0     1        1   6
      157       0     0        3   6
      157       0     1        1   6
      158       0     0        3   6
      158       0     1        1   6
      159       0     0        3   6
      159       0     1        1   6
      160       0     0        3   6
      160       0     1        1   6
      161       0     0        3   9
      161       0     1        2   9
      162       0     0        3   9
      162       0     1        2   9
      163       0     0        3   9
      163       0     1        2   9
      164       0     0        3   9
      164       0     1        2   9
      165       0     0        3   9
      165       0     1        2   9
      166       0     0        3   9
      166       0     1        2   9
      167       0     0        3   9
      167       0     1        2   9
      168       0     0        3   9
      168       0     1        2   9
      169       0     0        3   9
      169       0     1        2   9
      170       0     0        3  18
      170       0     1        3  18
      171       0     0        3  18
      171       0     1        3  18
      172       0     0        3  18
      172       0     1        3  18
      173       0     0        3  18
      173       0     1        3  18
      174       0     0        3  18
      174       0     1        3  18
      175       0     0        3  18
      175       0     1        3  18
      176       0     0        3  18
      176       0     1        3  18
      177       0     0        3  18
      177       0     1        3  18
      178       0     0        3  18
      178       0     1        3  18
      179       0     0        3  18
      179       0     1        3  18
      180       0     0        3  18
      180       0     1        3  18
      181       0     0        3  18
      181       0     1        3  18
      182       0     0        3  18
      182       0     1        3  18
      183       0     0        3  18
      183       0     1        3  18
      184       0     0        3  18
      184       0     1        3  18
      185       0     0        3  18
      185       0     1        3  18
      186       0     0        3  18
      186       0     1        3  18
      187       0     0        3  18
      187       0     1        3  18
      188       0     0        3   2
      188       0     1        4   2
      189       0     0        3   2
      189       0     1        4   2
      190       0     0        4   4
      190       0     1        1   4
      191       0     0        4   4
      191       0     1        1   4
      192       0     0        4   4
      192       0     1        1   4
      193       0     0        4   4
      193       0     1        1   4
      194       0     0        4  11
      194       0     1        2  11
      195       0     0        4  11
      195       0     1        2  11
      196       0     0        4  11
      196       0     1        2  11
      197       0     0        4  11
      197       0     1        2  11
      198       0     0        4  11
      198       0     1        2  11
      199       0     0        4  11
      199       0     1        2  11
      200       0     0        4  11
      200       0     1        2  11
      201       0     0        4  11
      201       0     1        2  11
      202       0     0        4  11
      202       0     1        2  11
      203       0     0        4  11
      203       0     1        2  11
      204       0     0        4  11
      204       0     1        2  11
      205       0     0        4  14
      205       0     1        3  14
      206       0     0        4  14
      206       0     1        3  14
      207       0     0        4  14
      207       0     1        3  14
      208       0     0        4  14
      208       0     1        3  14
      209       0     0        4  14
      209       0     1        3  14
      210       0     0        4  14
      210       0     1        3  14
      211       0     0        4  14
      211       0     1        3  14
      212       0     0        4  14
      212       0     1        3  14
      213       0     0        4  14
      213       0     1        3  14
      214       0     0        4  14
      214       0     1        3  14
      215       0     0        4  14
      215       0     1        3  14
      216       0     0        4  14
      216       0     1        3  14
      217       0     0        4  14
      217       0     1        3  14
      218       0     0        4  14
      218       0     1        3  14
      219       0     0        4  22
      219       0     1        4  22
      220       0     0        4  22
      220       0     1        4  22
      221       0     0        4  22
      221       0     1        4  22
      222       0     0        4  22
      222       0     1        4  22
      223       0     0        4  22
      223       0     1        4  22
      224       0     0        4  22
      224       0     1        4  22
      225       0     0        4  22
      225       0     1        4  22
      226       0     0        4  22
      226       0     1        4  22
      227       0     0        4  22
      227       0     1        4  22
      228       0     0        4  22
      228       0     1        4  22
      229       0     0        4  22
      229       0     1        4  22
      230       0     0        4  22
      230       0     1        4  22
      231       0     0        4  22
      231       0     1        4  22
      232       0     0        4  22
      232       0     1        4  22
      233       0     0        4  22
      233       0     1        4  22
      234       0     0        4  22
      234       0     1        4  22
      235       0     0        4  22
      235       0     1        4  22
      236       0     0        4  22
      236       0     1        4  22
      237       0     0        4  22
      237       0     1        4  22
      238       0     0        4  22
      238       0     1        4  22
      239       0     0        4  22
      239       0     1        4  22
      127       0     0        4  22
      127       0     1        4  22

24. Presidential election poll data set of Table 13.2


state pi n      x proportion
AK .379  5      3  0.6000000
AL .387 29      9  0.3103448
AR .389 17      2  0.1176471
AZ .449 35     13  0.3714286
CA .609 207   129  0.6231884
CO .537 37     16  0.4324324
CT .606 25     14  0.5600000
DC .925  4      4  1.0000000
DE .619  6      4  0.6666667
FL .509 128    73  0.5703125
GA .469 60     27  0.4500000
HI .718 7       6  0.8571429
IA .539 23      9  0.3913043
ID .359 10      1  0.1000000
IL .618 84     45  0.5357143
IN .498 42     20  0.4761905
KS .415 19      8  0.4210526
KY .412 28     10  0.3571429
LA .399 30     11  0.3666667
MA .618 47     20  0.4255319
MD .619 40     29  0.7250000
ME .577 11      9  0.8181818
MI .573 76     42  0.5526316
MN .541 44     22  0.5000000
MO .492 45     25  0.5555556
MS .430 20     11  0.5500000
MT .471 7       3  0.4285714
NC .497 66     23  0.3484848
ND .445 5       2  0.4000000
NE .416 12     10  0.8333333
NH .541 11      3  0.2727273
NJ .571 59     32  0.5423729
NM .569 13      7  0.5384615
NV .552 15      7  0.4666667
NY .629 116    77  0.6637931
OH .514 87     48  0.5517241
OK .344 22      4  0.1818182
OR .568 28     14  0.5000000
PA .545 92     53  0.5760870
RI .629 7       6  0.8571429
SC .449 29      9  0.3103448
SD .448 6       4  0.6666667
TN .418 40     17  0.4250000
TX .436 123    61  0.4959350
UT .342 15     10  0.6666667
VA .526 57     28  0.4912281
VT .675 5       4  0.8000000
WA .573 46     31  0.6739130
WI .562 45     26  0.5777778
WV .425 11      6  0.5454545
WY .325 4       2  0.5000000        

25. Attitudes about abortion data set of Table 13.3, with data shown at the individual level


     gender response question case
       1        1        1    1
       1        1        2    1
       1        1        3    1
       1        1        1    2
       1        1        2    2
       1        1        3    2
       1        1        1    3
       1        1        2    3
       1        1        3    3
       1        1        1    4
       1        1        2    4
       1        1        3    4
       1        1        1    5
       1        1        2    5
       1        1        3    5
       1        1        1    6
       1        1        2    6
       1        1        3    6
       1        1        1    7
       1        1        2    7
       1        1        3    7
       1        1        1    8
       1        1        2    8
       1        1        3    8
       1        1        1    9
       1        1        2    9
       1        1        3    9
       1        1        1   10
       1        1        2   10
       1        1        3   10
       1        1        1   11
       1        1        2   11
       1        1        3   11
       1        1        1   12
       1        1        2   12
       1        1        3   12
       1        1        1   13
       1        1        2   13
       1        1        3   13
       1        1        1   14
       1        1        2   14
       1        1        3   14
       1        1        1   15
       1        1        2   15
       1        1        3   15
       1        1        1   16
       1        1        2   16
       1        1        3   16
       1        1        1   17
       1        1        2   17
       1        1        3   17
       1        1        1   18
       1        1        2   18
       1        1        3   18
       1        1        1   19
       1        1        2   19
       1        1        3   19
       1        1        1   20
       1        1        2   20
       1        1        3   20
       1        1        1   21
       1        1        2   21
       1        1        3   21
       1        1        1   22
       1        1        2   22
       1        1        3   22
       1        1        1   23
       1        1        2   23
       1        1        3   23
       1        1        1   24
       1        1        2   24
       1        1        3   24
       1        1        1   25
       1        1        2   25
       1        1        3   25
       1        1        1   26
       1        1        2   26
       1        1        3   26
       1        1        1   27
       1        1        2   27
       1        1        3   27
       1        1        1   28
       1        1        2   28
       1        1        3   28
       1        1        1   29
       1        1        2   29
       1        1        3   29
       1        1        1   30
       1        1        2   30
       1        1        3   30
       1        1        1   31
       1        1        2   31
       1        1        3   31
       1        1        1   32
       1        1        2   32
       1        1        3   32
       1        1        1   33
       1        1        2   33
       1        1        3   33
       1        1        1   34
       1        1        2   34
       1        1        3   34
       1        1        1   35
       1        1        2   35
       1        1        3   35
       1        1        1   36
       1        1        2   36
       1        1        3   36
       1        1        1   37
       1        1        2   37
       1        1        3   37
       1        1        1   38
       1        1        2   38
       1        1        3   38
       1        1        1   39
       1        1        2   39
       1        1        3   39
       1        1        1   40
       1        1        2   40
       1        1        3   40
       1        1        1   41
       1        1        2   41
       1        1        3   41
       1        1        1   42
       1        1        2   42
       1        1        3   42
       1        1        1   43
       1        1        2   43
       1        1        3   43
       1        1        1   44
       1        1        2   44
       1        1        3   44
       1        1        1   45
       1        1        2   45
       1        1        3   45
       1        1        1   46
       1        1        2   46
       1        1        3   46
       1        1        1   47
       1        1        2   47
       1        1        3   47
       1        1        1   48
       1        1        2   48
       1        1        3   48
       1        1        1   49
       1        1        2   49
       1        1        3   49
       1        1        1   50
       1        1        2   50
       1        1        3   50
       1        1        1   51
       1        1        2   51
       1        1        3   51
       1        1        1   52
       1        1        2   52
       1        1        3   52
       1        1        1   53
       1        1        2   53
       1        1        3   53
       1        1        1   54
       1        1        2   54
       1        1        3   54
       1        1        1   55
       1        1        2   55
       1        1        3   55
       1        1        1   56
       1        1        2   56
       1        1        3   56
       1        1        1   57
       1        1        2   57
       1        1        3   57
       1        1        1   58
       1        1        2   58
       1        1        3   58
       1        1        1   59
       1        1        2   59
       1        1        3   59
       1        1        1   60
       1        1        2   60
       1        1        3   60
       1        1        1   61
       1        1        2   61
       1        1        3   61
       1        1        1   62
       1        1        2   62
       1        1        3   62
       1        1        1   63
       1        1        2   63
       1        1        3   63
       1        1        1   64
       1        1        2   64
       1        1        3   64
       1        1        1   65
       1        1        2   65
       1        1        3   65
       1        1        1   66
       1        1        2   66
       1        1        3   66
       1        1        1   67
       1        1        2   67
       1        1        3   67
       1        1        1   68
       1        1        2   68
       1        1        3   68
       1        1        1   69
       1        1        2   69
       1        1        3   69
       1        1        1   70
       1        1        2   70
       1        1        3   70
       1        1        1   71
       1        1        2   71
       1        1        3   71
       1        1        1   72
       1        1        2   72
       1        1        3   72
       1        1        1   73
       1        1        2   73
       1        1        3   73
       1        1        1   74
       1        1        2   74
       1        1        3   74
       1        1        1   75
       1        1        2   75
       1        1        3   75
       1        1        1   76
       1        1        2   76
       1        1        3   76
       1        1        1   77
       1        1        2   77
       1        1        3   77
       1        1        1   78
       1        1        2   78
       1        1        3   78
       1        1        1   79
       1        1        2   79
       1        1        3   79
       1        1        1   80
       1        1        2   80
       1        1        3   80
       1        1        1   81
       1        1        2   81
       1        1        3   81
       1        1        1   82
       1        1        2   82
       1        1        3   82
       1        1        1   83
       1        1        2   83
       1        1        3   83
       1        1        1   84
       1        1        2   84
       1        1        3   84
       1        1        1   85
       1        1        2   85
       1        1        3   85
       1        1        1   86
       1        1        2   86
       1        1        3   86
       1        1        1   87
       1        1        2   87
       1        1        3   87
       1        1        1   88
       1        1        2   88
       1        1        3   88
       1        1        1   89
       1        1        2   89
       1        1        3   89
       1        1        1   90
       1        1        2   90
       1        1        3   90
       1        1        1   91
       1        1        2   91
       1        1        3   91
       1        1        1   92
       1        1        2   92
       1        1        3   92
       1        1        1   93
       1        1        2   93
       1        1        3   93
       1        1        1   94
       1        1        2   94
       1        1        3   94
       1        1        1   95
       1        1        2   95
       1        1        3   95
       1        1        1   96
       1        1        2   96
       1        1        3   96
       1        1        1   97
       1        1        2   97
       1        1        3   97
       1        1        1   98
       1        1        2   98
       1        1        3   98
       1        1        1   99
       1        1        2   99
       1        1        3   99
       1        1        1  100
       1        1        2  100
       1        1        3  100
       1        1        1  101
       1        1        2  101
       1        1        3  101
       1        1        1  102
       1        1        2  102
       1        1        3  102
       1        1        1  103
       1        1        2  103
       1        1        3  103
       1        1        1  104
       1        1        2  104
       1        1        3  104
       1        1        1  105
       1        1        2  105
       1        1        3  105
       1        1        1  106
       1        1        2  106
       1        1        3  106
       1        1        1  107
       1        1        2  107
       1        1        3  107
       1        1        1  108
       1        1        2  108
       1        1        3  108
       1        1        1  109
       1        1        2  109
       1        1        3  109
       1        1        1  110
       1        1        2  110
       1        1        3  110
       1        1        1  111
       1        1        2  111
       1        1        3  111
       1        1        1  112
       1        1        2  112
       1        1        3  112
       1        1        1  113
       1        1        2  113
       1        1        3  113
       1        1        1  114
       1        1        2  114
       1        1        3  114
       1        1        1  115
       1        1        2  115
       1        1        3  115
       1        1        1  116
       1        1        2  116
       1        1        3  116
       1        1        1  117
       1        1        2  117
       1        1        3  117
       1        1        1  118
       1        1        2  118
       1        1        3  118
       1        1        1  119
       1        1        2  119
       1        1        3  119
       1        1        1  120
       1        1        2  120
       1        1        3  120
       1        1        1  121
       1        1        2  121
       1        1        3  121
       1        1        1  122
       1        1        2  122
       1        1        3  122
       1        1        1  123
       1        1        2  123
       1        1        3  123
       1        1        1  124
       1        1        2  124
       1        1        3  124
       1        1        1  125
       1        1        2  125
       1        1        3  125
       1        1        1  126
       1        1        2  126
       1        1        3  126
       1        1        1  127
       1        1        2  127
       1        1        3  127
       1        1        1  128
       1        1        2  128
       1        1        3  128
       1        1        1  129
       1        1        2  129
       1        1        3  129
       1        1        1  130
       1        1        2  130
       1        1        3  130
       1        1        1  131
       1        1        2  131
       1        1        3  131
       1        1        1  132
       1        1        2  132
       1        1        3  132
       1        1        1  133
       1        1        2  133
       1        1        3  133
       1        1        1  134
       1        1        2  134
       1        1        3  134
       1        1        1  135
       1        1        2  135
       1        1        3  135
       1        1        1  136
       1        1        2  136
       1        1        3  136
       1        1        1  137
       1        1        2  137
       1        1        3  137
       1        1        1  138
       1        1        2  138
       1        1        3  138
       1        1        1  139
       1        1        2  139
       1        1        3  139
       1        1        1  140
       1        1        2  140
       1        1        3  140
       1        1        1  141
       1        1        2  141
       1        1        3  141
       1        1        1  142
       1        1        2  142
       1        1        3  142
       1        1        1  143
       1        1        2  143
       1        1        3  143
       1        1        1  144
       1        1        2  144
       1        1        3  144
       1        1        1  145
       1        1        2  145
       1        1        3  145
       1        1        1  146
       1        1        2  146
       1        1        3  146
       1        1        1  147
       1        1        2  147
       1        1        3  147
       1        1        1  148
       1        1        2  148
       1        1        3  148
       1        1        1  149
       1        1        2  149
       1        1        3  149
       1        1        1  150
       1        1        2  150
       1        1        3  150
       1        1        1  151
       1        1        2  151
       1        1        3  151
       1        1        1  152
       1        1        2  152
       1        1        3  152
       1        1        1  153
       1        1        2  153
       1        1        3  153
       1        1        1  154
       1        1        2  154
       1        1        3  154
       1        1        1  155
       1        1        2  155
       1        1        3  155
       1        1        1  156
       1        1        2  156
       1        1        3  156
       1        1        1  157
       1        1        2  157
       1        1        3  157
       1        1        1  158
       1        1        2  158
       1        1        3  158
       1        1        1  159
       1        1        2  159
       1        1        3  159
       1        1        1  160
       1        1        2  160
       1        1        3  160
       1        1        1  161
       1        1        2  161
       1        1        3  161
       1        1        1  162
       1        1        2  162
       1        1        3  162
       1        1        1  163
       1        1        2  163
       1        1        3  163
       1        1        1  164
       1        1        2  164
       1        1        3  164
       1        1        1  165
       1        1        2  165
       1        1        3  165
       1        1        1  166
       1        1        2  166
       1        1        3  166
       1        1        1  167
       1        1        2  167
       1        1        3  167
       1        1        1  168
       1        1        2  168
       1        1        3  168
       1        1        1  169
       1        1        2  169
       1        1        3  169
       1        1        1  170
       1        1        2  170
       1        1        3  170
       1        1        1  171
       1        1        2  171
       1        1        3  171
       1        1        1  172
       1        1        2  172
       1        1        3  172
       1        1        1  173
       1        1        2  173
       1        1        3  173
       1        1        1  174
       1        1        2  174
       1        1        3  174
       1        1        1  175
       1        1        2  175
       1        1        3  175
       1        1        1  176
       1        1        2  176
       1        1        3  176
       1        1        1  177
       1        1        2  177
       1        1        3  177
       1        1        1  178
       1        1        2  178
       1        1        3  178
       1        1        1  179
       1        1        2  179
       1        1        3  179
       1        1        1  180
       1        1        2  180
       1        1        3  180
       1        1        1  181
       1        1        2  181
       1        1        3  181
       1        1        1  182
       1        1        2  182
       1        1        3  182
       1        1        1  183
       1        1        2  183
       1        1        3  183
       1        1        1  184
       1        1        2  184
       1        1        3  184
       1        1        1  185
       1        1        2  185
       1        1        3  185
       1        1        1  186
       1        1        2  186
       1        1        3  186
       1        1        1  187
       1        1        2  187
       1        1        3  187
       1        1        1  188
       1        1        2  188
       1        1        3  188
       1        1        1  189
       1        1        2  189
       1        1        3  189
       1        1        1  190
       1        1        2  190
       1        1        3  190
       1        1        1  191
       1        1        2  191
       1        1        3  191
       1        1        1  192
       1        1        2  192
       1        1        3  192
       1        1        1  193
       1        1        2  193
       1        1        3  193
       1        1        1  194
       1        1        2  194
       1        1        3  194
       1        1        1  195
       1        1        2  195
       1        1        3  195
       1        1        1  196
       1        1        2  196
       1        1        3  196
       1        1        1  197
       1        1        2  197
       1        1        3  197
       1        1        1  198
       1        1        2  198
       1        1        3  198
       1        1        1  199
       1        1        2  199
       1        1        3  199
       1        1        1  200
       1        1        2  200
       1        1        3  200
       1        1        1  201
       1        1        2  201
       1        1        3  201
       1        1        1  202
       1        1        2  202
       1        1        3  202
       1        1        1  203
       1        1        2  203
       1        1        3  203
       1        1        1  204
       1        1        2  204
       1        1        3  204
       1        1        1  205
       1        1        2  205
       1        1        3  205
       1        1        1  206
       1        1        2  206
       1        1        3  206
       1        1        1  207
       1        1        2  207
       1        1        3  207
       1        1        1  208
       1        1        2  208
       1        1        3  208
       1        1        1  209
       1        1        2  209
       1        1        3  209
       1        1        1  210
       1        1        2  210
       1        1        3  210
       1        1        1  211
       1        1        2  211
       1        1        3  211
       1        1        1  212
       1        1        2  212
       1        1        3  212
       1        1        1  213
       1        1        2  213
       1        1        3  213
       1        1        1  214
       1        1        2  214
       1        1        3  214
       1        1        1  215
       1        1        2  215
       1        1        3  215
       1        1        1  216
       1        1        2  216
       1        1        3  216
       1        1        1  217
       1        1        2  217
       1        1        3  217
       1        1        1  218
       1        1        2  218
       1        1        3  218
       1        1        1  219
       1        1        2  219
       1        1        3  219
       1        1        1  220
       1        1        2  220
       1        1        3  220
       1        1        1  221
       1        1        2  221
       1        1        3  221
       1        1        1  222
       1        1        2  222
       1        1        3  222
       1        1        1  223
       1        1        2  223
       1        1        3  223
       1        1        1  224
       1        1        2  224
       1        1        3  224
       1        1        1  225
       1        1        2  225
       1        1        3  225
       1        1        1  226
       1        1        2  226
       1        1        3  226
       1        1        1  227
       1        1        2  227
       1        1        3  227
       1        1        1  228
       1        1        2  228
       1        1        3  228
       1        1        1  229
       1        1        2  229
       1        1        3  229
       1        1        1  230
       1        1        2  230
       1        1        3  230
       1        1        1  231
       1        1        2  231
       1        1        3  231
       1        1        1  232
       1        1        2  232
       1        1        3  232
       1        1        1  233
       1        1        2  233
       1        1        3  233
       1        1        1  234
       1        1        2  234
       1        1        3  234
       1        1        1  235
       1        1        2  235
       1        1        3  235
       1        1        1  236
       1        1        2  236
       1        1        3  236
       1        1        1  237
       1        1        2  237
       1        1        3  237
       1        1        1  238
       1        1        2  238
       1        1        3  238
       1        1        1  239
       1        1        2  239
       1        1        3  239
       1        1        1  240
       1        1        2  240
       1        1        3  240
       1        1        1  241
       1        1        2  241
       1        1        3  241
       1        1        1  242
       1        1        2  242
       1        1        3  242
       1        1        1  243
       1        1        2  243
       1        1        3  243
       1        1        1  244
       1        1        2  244
       1        1        3  244
       1        1        1  245
       1        1        2  245
       1        1        3  245
       1        1        1  246
       1        1        2  246
       1        1        3  246
       1        1        1  247
       1        1        2  247
       1        1        3  247
       1        1        1  248
       1        1        2  248
       1        1        3  248
       1        1        1  249
       1        1        2  249
       1        1        3  249
       1        1        1  250
       1        1        2  250
       1        1        3  250
       1        1        1  251
       1        1        2  251
       1        1        3  251
       1        1        1  252
       1        1        2  252
       1        1        3  252
       1        1        1  253
       1        1        2  253
       1        1        3  253
       1        1        1  254
       1        1        2  254
       1        1        3  254
       1        1        1  255
       1        1        2  255
       1        1        3  255
       1        1        1  256
       1        1        2  256
       1        1        3  256
       1        1        1  257
       1        1        2  257
       1        1        3  257
       1        1        1  258
       1        1        2  258
       1        1        3  258
       1        1        1  259
       1        1        2  259
       1        1        3  259
       1        1        1  260
       1        1        2  260
       1        1        3  260
       1        1        1  261
       1        1        2  261
       1        1        3  261
       1        1        1  262
       1        1        2  262
       1        1        3  262
       1        1        1  263
       1        1        2  263
       1        1        3  263
       1        1        1  264
       1        1        2  264
       1        1        3  264
       1        1        1  265
       1        1        2  265
       1        1        3  265
       1        1        1  266
       1        1        2  266
       1        1        3  266
       1        1        1  267
       1        1        2  267
       1        1        3  267
       1        1        1  268
       1        1        2  268
       1        1        3  268
       1        1        1  269
       1        1        2  269
       1        1        3  269
       1        1        1  270
       1        1        2  270
       1        1        3  270
       1        1        1  271
       1        1        2  271
       1        1        3  271
       1        1        1  272
       1        1        2  272
       1        1        3  272
       1        1        1  273
       1        1        2  273
       1        1        3  273
       1        1        1  274
       1        1        2  274
       1        1        3  274
       1        1        1  275
       1        1        2  275
       1        1        3  275
       1        1        1  276
       1        1        2  276
       1        1        3  276
       1        1        1  277
       1        1        2  277
       1        1        3  277
       1        1        1  278
       1        1        2  278
       1        1        3  278
       1        1        1  279
       1        1        2  279
       1        1        3  279
       1        1        1  280
       1        1        2  280
       1        1        3  280
       1        1        1  281
       1        1        2  281
       1        1        3  281
       1        1        1  282
       1        1        2  282
       1        1        3  282
       1        1        1  283
       1        1        2  283
       1        1        3  283
       1        1        1  284
       1        1        2  284
       1        1        3  284
       1        1        1  285
       1        1        2  285
       1        1        3  285
       1        1        1  286
       1        1        2  286
       1        1        3  286
       1        1        1  287
       1        1        2  287
       1        1        3  287
       1        1        1  288
       1        1        2  288
       1        1        3  288
       1        1        1  289
       1        1        2  289
       1        1        3  289
       1        1        1  290
       1        1        2  290
       1        1        3  290
       1        1        1  291
       1        1        2  291
       1        1        3  291
       1        1        1  292
       1        1        2  292
       1        1        3  292
       1        1        1  293
       1        1        2  293
       1        1        3  293
       1        1        1  294
       1        1        2  294
       1        1        3  294
       1        1        1  295
       1        1        2  295
       1        1        3  295
       1        1        1  296
       1        1        2  296
       1        1        3  296
       1        1        1  297
       1        1        2  297
       1        1        3  297
       1        1        1  298
       1        1        2  298
       1        1        3  298
       1        1        1  299
       1        1        2  299
       1        1        3  299
       1        1        1  300
       1        1        2  300
       1        1        3  300
       1        1        1  301
       1        1        2  301
       1        1        3  301
       1        1        1  302
       1        1        2  302
       1        1        3  302
       1        1        1  303
       1        1        2  303
       1        1        3  303
       1        1        1  304
       1        1        2  304
       1        1        3  304
       1        1        1  305
       1        1        2  305
       1        1        3  305
       1        1        1  306
       1        1        2  306
       1        1        3  306
       1        1        1  307
       1        1        2  307
       1        1        3  307
       1        1        1  308
       1        1        2  308
       1        1        3  308
       1        1        1  309
       1        1        2  309
       1        1        3  309
       1        1        1  310
       1        1        2  310
       1        1        3  310
       1        1        1  311
       1        1        2  311
       1        1        3  311
       1        1        1  312
       1        1        2  312
       1        1        3  312
       1        1        1  313
       1        1        2  313
       1        1        3  313
       1        1        1  314
       1        1        2  314
       1        1        3  314
       1        1        1  315
       1        1        2  315
       1        1        3  315
       1        1        1  316
       1        1        2  316
       1        1        3  316
       1        1        1  317
       1        1        2  317
       1        1        3  317
       1        1        1  318
       1        1        2  318
       1        1        3  318
       1        1        1  319
       1        1        2  319
       1        1        3  319
       1        1        1  320
       1        1        2  320
       1        1        3  320
       1        1        1  321
       1        1        2  321
       1        1        3  321
       1        1        1  322
       1        1        2  322
       1        1        3  322
       1        1        1  323
       1        1        2  323
       1        1        3  323
       1        1        1  324
       1        1        2  324
       1        1        3  324
       1        1        1  325
       1        1        2  325
       1        1        3  325
       1        1        1  326
       1        1        2  326
       1        1        3  326
       1        1        1  327
       1        1        2  327
       1        1        3  327
       1        1        1  328
       1        1        2  328
       1        1        3  328
       1        1        1  329
       1        1        2  329
       1        1        3  329
       1        1        1  330
       1        1        2  330
       1        1        3  330
       1        1        1  331
       1        1        2  331
       1        1        3  331
       1        1        1  332
       1        1        2  332
       1        1        3  332
       1        1        1  333
       1        1        2  333
       1        1        3  333
      1        1        1  334
      1        1        2  334
      1        1        3  334
      1        1        1  335
      1        1        2  335
      1        1        3  335
      1        1        1  336
      1        1        2  336
      1        1        3  336
      1        1        1  337
      1        1        2  337
      1        1        3  337
      1        1        1  338
      1        1        2  338
      1        1        3  338
      1        1        1  339
      1        1        2  339
      1        1        3  339
      1        1        1  340
      1        1        2  340
      1        1        3  340
      1        1        1  341
      1        1        2  341
      1        1        3  341
      1        1        1  342
      1        1        2  342
      1        1        3  342
      1        1        1  343
      1        1        2  343
      1        1        3  343
      1        1        1  344
      1        1        2  344
      1        1        3  344
      1        1        1  345
      1        1        2  345
      1        1        3  345
      1        1        1  346
      1        1        2  346
      1        1        3  346
      1        1        1  347
      1        1        2  347
      1        1        3  347
      1        1        1  348
      1        1        2  348
      1        1        3  348
      1        1        1  349
      1        1        2  349
      1        1        3  349
      1        1        1  350
      1        1        2  350
      1        1        3  350
      1        1        1  351
      1        1        2  351
      1        1        3  351
      1        1        1  352
      1        1        2  352
      1        1        3  352
      1        1        1  353
      1        1        2  353
      1        1        3  353
      1        1        1  354
      1        1        2  354
      1        1        3  354
      1        1        1  355
      1        1        2  355
      1        1        3  355
      1        1        1  356
      1        1        2  356
      1        1        3  356
      1        1        1  357
      1        1        2  357
      1        1        3  357
      1        1        1  358
      1        1        2  358
      1        1        3  358
      1        1        1  359
      1        1        2  359
      1        1        3  359
      1        1        1  360
      1        1        2  360
      1        1        3  360
      1        1        1  361
      1        1        2  361
      1        1        3  361
      1        1        1  362
      1        1        2  362
      1        1        3  362
      1        1        1  363
      1        1        2  363
      1        1        3  363
      1        1        1  364
      1        1        2  364
      1        1        3  364
      1        1        1  365
      1        1        2  365
      1        1        3  365
      1        1        1  366
      1        1        2  366
      1        1        3  366
      1        1        1  367
      1        1        2  367
      1        1        3  367
      1        1        1  368
      1        1        2  368
      1        1        3  368
      1        1        1  369
      1        1        2  369
      1        1        3  369
      1        1        1  370
      1        1        2  370
      1        1        3  370
      1        1        1  371
      1        1        2  371
      1        1        3  371
      1        1        1  372
      1        1        2  372
      1        1        3  372
      1        1        1  373
      1        1        2  373
      1        1        3  373
      1        1        1  374
      1        1        2  374
      1        1        3  374
      1        1        1  375
      1        1        2  375
      1        1        3  375
      1        1        1  376
      1        1        2  376
      1        1        3  376
      1        1        1  377
      1        1        2  377
      1        1        3  377
      1        1        1  378
      1        1        2  378
      1        1        3  378
      1        1        1  379
      1        1        2  379
      1        1        3  379
      1        1        1  380
      1        1        2  380
      1        1        3  380
      1        1        1  381
      1        1        2  381
      1        1        3  381
      1        1        1  382
      1        1        2  382
      1        1        3  382
      1        1        1  383
      1        1        2  383
      1        1        3  383
      1        1        1  384
      1        1        2  384
      1        1        3  384
      1        1        1  385
      1        1        2  385
      1        1        3  385
      1        1        1  386
      1        1        2  386
      1        1        3  386
      1        1        1  387
      1        1        2  387
      1        1        3  387
      1        1        1  388
      1        1        2  388
      1        1        3  388
      1        1        1  389
      1        1        2  389
      1        1        3  389
      1        1        1  390
      1        1        2  390
      1        1        3  390
      1        1        1  391
      1        1        2  391
      1        1        3  391
      1        1        1  392
      1        1        2  392
      1        1        3  392
      1        1        1  393
      1        1        2  393
      1        1        3  393
      1        1        1  394
      1        1        2  394
      1        1        3  394
      1        1        1  395
      1        1        2  395
      1        1        3  395
      1        1        1  396
      1        1        2  396
      1        1        3  396
      1        1        1  397
      1        1        2  397
      1        1        3  397
      1        1        1  398
      1        1        2  398
      1        1        3  398
      1        1        1  399
      1        1        2  399
      1        1        3  399
      1        1        1  400
      1        1        2  400
      1        1        3  400
      1        1        1  401
      1        1        2  401
      1        1        3  401
      1        1        1  402
      1        1        2  402
      1        1        3  402
      1        1        1  403
      1        1        2  403
      1        1        3  403
      1        1        1  404
      1        1        2  404
      1        1        3  404
      1        1        1  405
      1        1        2  405
      1        1        3  405
      1        1        1  406
      1        1        2  406
      1        1        3  406
      1        1        1  407
      1        1        2  407
      1        1        3  407
      1        1        1  408
      1        1        2  408
      1        1        3  408
      1        1        1  409
      1        1        2  409
      1        1        3  409
      1        1        1  410
      1        1        2  410
      1        1        3  410
      1        1        1  411
      1        1        2  411
      1        1        3  411
      1        1        1  412
      1        1        2  412
      1        1        3  412
      1        1        1  413
      1        1        2  413
      1        1        3  413
      1        1        1  414
      1        1        2  414
      1        1        3  414
      1        1        1  415
      1        1        2  415
      1        1        3  415
      1        1        1  416
      1        1        2  416
      1        1        3  416
      1        1        1  417
      1        1        2  417
      1        1        3  417
      1        1        1  418
      1        1        2  418
      1        1        3  418
      1        1        1  419
      1        1        2  419
      1        1        3  419
      1        1        1  420
      1        1        2  420
      1        1        3  420
      1        1        1  421
      1        1        2  421
      1        1        3  421
      1        1        1  422
      1        1        2  422
      1        1        3  422
      1        1        1  423
      1        1        2  423
      1        1        3  423
      1        1        1  424
      1        1        2  424
      1        1        3  424
      1        1        1  425
      1        1        2  425
      1        1        3  425
      1        1        1  426
      1        1        2  426
      1        1        3  426
      1        1        1  427
      1        1        2  427
      1        1        3  427
      1        1        1  428
      1        1        2  428
      1        1        3  428
      1        1        1  429
      1        1        2  429
      1        1        3  429
      1        1        1  430
      1        1        2  430
      1        1        3  430
      1        1        1  431
      1        1        2  431
      1        1        3  431
      1        1        1  432
      1        1        2  432
      1        1        3  432
      1        1        1  433
      1        1        2  433
      1        1        3  433
      1        1        1  434
      1        1        2  434
      1        1        3  434
      1        1        1  435
      1        1        2  435
      1        1        3  435
      1        1        1  436
      1        1        2  436
      1        1        3  436
      1        1        1  437
      1        1        2  437
      1        1        3  437
      1        1        1  438
      1        1        2  438
      1        1        3  438
      1        1        1  439
      1        1        2  439
      1        1        3  439
      1        1        1  440
      1        1        2  440
      1        1        3  440
      1        1        1  441
      1        1        2  441
      1        0        3  441
      1        1        1  442
      1        1        2  442
      1        0        3  442
      1        1        1  443
      1        1        2  443
      1        0        3  443
      1        1        1  444
      1        1        2  444
      1        0        3  444
      1        1        1  445
      1        1        2  445
      1        0        3  445
      1        1        1  446
      1        1        2  446
      1        0        3  446
      1        1        1  447
      1        1        2  447
      1        0        3  447
      1        1        1  448
      1        1        2  448
      1        0        3  448
      1        1        1  449
      1        1        2  449
      1        0        3  449
      1        1        1  450
      1        1        2  450
      1        0        3  450
      1        1        1  451
      1        1        2  451
      1        0        3  451
      1        1        1  452
      1        1        2  452
      1        0        3  452
      1        1        1  453
      1        1        2  453
      1        0        3  453
      1        1        1  454
      1        1        2  454
      1        0        3  454
      1        1        1  455
      1        1        2  455
      1        0        3  455
      1        1        1  456
      1        1        2  456
      1        0        3  456
      1        1        1  457
      1        1        2  457
      1        0        3  457
      1        1        1  458
      1        1        2  458
      1        0        3  458
      1        1        1  459
      1        1        2  459
      1        0        3  459
      1        1        1  460
      1        1        2  460
      1        0        3  460
      1        1        1  461
      1        1        2  461
      1        0        3  461
      1        1        1  462
      1        1        2  462
      1        0        3  462
      1        1        1  463
      1        1        2  463
      1        0        3  463
      1        1        1  464
      1        1        2  464
      1        0        3  464
      1        1        1  465
      1        1        2  465
      1        0        3  465
      1        0        1  466
      1        1        2  466
      1        1        3  466
      1        0        1  467
      1        1        2  467
      1        1        3  467
      1        0        1  468
      1        1        2  468
      1        1        3  468
      1        0        1  469
      1        1        2  469
      1        1        3  469
      1        0        1  470
      1        1        2  470
      1        1        3  470
      1        0        1  471
      1        1        2  471
      1        1        3  471
      1        0        1  472
      1        1        2  472
      1        1        3  472
      1        0        1  473
      1        1        2  473
      1        1        3  473
      1        0        1  474
      1        1        2  474
      1        1        3  474
      1        0        1  475
      1        1        2  475
      1        1        3  475
      1        0        1  476
      1        1        2  476
      1        1        3  476
      1        0        1  477
      1        1        2  477
      1        1        3  477
      1        0        1  478
      1        1        2  478
      1        1        3  478
      1        0        1  479
      1        1        2  479
      1        1        3  479
      1        0        1  480
      1        1        2  480
      1        0        3  480
      1        0        1  481
      1        1        2  481
      1        0        3  481
      1        0        1  482
      1        1        2  482
      1        0        3  482
      1        0        1  483
      1        1        2  483
      1        0        3  483
      1        0        1  484
      1        1        2  484
      1        0        3  484
      1        0        1  485
      1        1        2  485
      1        0        3  485
      1        0        1  486
      1        1        2  486
      1        0        3  486
      1        0        1  487
      1        1        2  487
      1        0        3  487
      1        0        1  488
      1        1        2  488
      1        0        3  488
      1        0        1  489
      1        1        2  489
      1        0        3  489
      1        0        1  490
      1        1        2  490
      1        0        3  490
      1        0        1  491
      1        1        2  491
      1        0        3  491
      1        0        1  492
      1        1        2  492
      1        0        3  492
      1        0        1  493
      1        1        2  493
      1        0        3  493
      1        0        1  494
      1        1        2  494
      1        0        3  494
      1        0        1  495
      1        1        2  495
      1        0        3  495
      1        0        1  496
      1        1        2  496
      1        0        3  496
      1        0        1  497
      1        1        2  497
      1        0        3  497
      1        1        1  498
      1        0        2  498
      1        1        3  498
      1        1        1  499
      1        0        2  499
      1        1        3  499
      1        1        1  500
      1        0        2  500
      1        1        3  500
      1        1        1  501
      1        0        2  501
      1        1        3  501
      1        1        1  502
      1        0        2  502
      1        1        3  502
      1        1        1  503
      1        0        2  503
      1        1        3  503
      1        1        1  504
      1        0        2  504
      1        1        3  504
      1        1        1  505
      1        0        2  505
      1        1        3  505
      1        1        1  506
      1        0        2  506
      1        1        3  506
      1        1        1  507
      1        0        2  507
      1        1        3  507
      1        1        1  508
      1        0        2  508
      1        1        3  508
      1        1        1  509
      1        0        2  509
      1        1        3  509
      1        1        1  510
      1        0        2  510
      1        1        3  510
      1        1        1  511
      1        0        2  511
      1        1        3  511
      1        1        1  512
      1        0        2  512
      1        0        3  512
      1        1        1  513
      1        0        2  513
      1        0        3  513
      1        1        1  514
      1        0        2  514
      1        0        3  514
      1        1        1  515
      1        0        2  515
      1        0        3  515
      1        1        1  516
      1        0        2  516
      1        0        3  516
      1        1        1  517
      1        0        2  517
      1        0        3  517
      1        1        1  518
      1        0        2  518
      1        0        3  518
      1        1        1  519
      1        0        2  519
      1        0        3  519
      1        1        1  520
      1        0        2  520
      1        0        3  520
      1        1        1  521
      1        0        2  521
      1        0        3  521
      1        1        1  522
      1        0        2  522
      1        0        3  522
      1        1        1  523
      1        0        2  523
      1        0        3  523
      1        1        1  524
      1        0        2  524
      1        0        3  524
      1        1        1  525
      1        0        2  525
      1        0        3  525
      1        1        1  526
      1        0        2  526
      1        0        3  526
      1        1        1  527
      1        0        2  527
      1        0        3  527
      1        1        1  528
      1        0        2  528
      1        0        3  528
      1        1        1  529
      1        0        2  529
      1        0        3  529
      1        1        1  530
      1        0        2  530
      1        0        3  530
      1        1        1  531
      1        0        2  531
      1        0        3  531
      1        1        1  532
      1        0        2  532
      1        0        3  532
      1        1        1  533
      1        0        2  533
      1        0        3  533
      1        1        1  534
      1        0        2  534
      1        0        3  534
      1        1        1  535
      1        0        2  535
      1        0        3  535
      1        1        1  536
      1        0        2  536
      1        0        3  536
      1        1        1  537
      1        0        2  537
      1        0        3  537
      1        1        1  538
      1        0        2  538
      1        0        3  538
      1        1        1  539
      1        0        2  539
      1        0        3  539
      1        1        1  540
      1        0        2  540
      1        0        3  540
      1        1        1  541
      1        0        2  541
      1        0        3  541
      1        1        1  542
      1        0        2  542
      1        0        3  542
      1        1        1  543
      1        0        2  543
      1        0        3  543
      1        1        1  544
      1        0        2  544
      1        0        3  544
      1        1        1  545
      1        0        2  545
      1        0        3  545
      1        1        1  546
      1        0        2  546
      1        0        3  546
      1        1        1  547
      1        0        2  547
      1        0        3  547
      1        1        1  548
      1        0        2  548
      1        0        3  548
      1        1        1  549
      1        0        2  549
      1        0        3  549
      1        1        1  550
      1        0        2  550
      1        0        3  550
      1        1        1  551
      1        0        2  551
      1        0        3  551
      1        1        1  552
      1        0        2  552
      1        0        3  552
      1        1        1  553
      1        0        2  553
      1        0        3  553
      1        1        1  554
      1        0        2  554
      1        0        3  554
      1        1        1  555
      1        0        2  555
      1        0        3  555
      1        1        1  556
      1        0        2  556
      1        0        3  556
      1        1        1  557
      1        0        2  557
      1        0        3  557
      1        1        1  558
      1        0        2  558
      1        0        3  558
      1        0        1  559
      1        0        2  559
      1        1        3  559
      1        0        1  560
      1        0        2  560
      1        1        3  560
      1        0        1  561
      1        0        2  561
      1        1        3  561
      1        0        1  562
      1        0        2  562
      1        1        3  562
      1        0        1  563
      1        0        2  563
      1        1        3  563
      1        0        1  564
      1        0        2  564
      1        1        3  564
      1        0        1  565
      1        0        2  565
      1        1        3  565
      1        0        1  566
      1        0        2  566
      1        1        3  566
      1        0        1  567
      1        0        2  567
      1        1        3  567
      1        0        1  568
      1        0        2  568
      1        1        3  568
      1        0        1  569
      1        0        2  569
      1        1        3  569
      1        0        1  570
      1        0        2  570
      1        1        3  570
      1        0        1  571
      1        0        2  571
      1        1        3  571
      1        0        1  572
      1        0        2  572
      1        1        3  572
      1        0        1  573
      1        0        2  573
      1        1        3  573
      1        0        1  574
      1        0        2  574
      1        1        3  574
      1        0        1  575
      1        0        2  575
      1        1        3  575
      1        0        1  576
      1        0        2  576
      1        1        3  576
      1        0        1  577
      1        0        2  577
      1        1        3  577
      1        0        1  578
      1        0        2  578
      1        1        3  578
      1        0        1  579
      1        0        2  579
      1        1        3  579
      1        0        1  580
      1        0        2  580
      1        1        3  580
      1        0        1  581
      1        0        2  581
      1        0        3  581
      1        0        1  582
      1        0        2  582
      1        0        3  582
      1        0        1  583
      1        0        2  583
      1        0        3  583
      1        0        1  584
      1        0        2  584
      1        0        3  584
      1        0        1  585
      1        0        2  585
      1        0        3  585
      1        0        1  586
      1        0        2  586
      1        0        3  586
      1        0        1  587
      1        0        2  587
      1        0        3  587
      1        0        1  588
      1        0        2  588
      1        0        3  588
      1        0        1  589
      1        0        2  589
      1        0        3  589
      1        0        1  590
      1        0        2  590
      1        0        3  590
      1        0        1  591
      1        0        2  591
      1        0        3  591
      1        0        1  592
      1        0        2  592
      1        0        3  592
      1        0        1  593
      1        0        2  593
      1        0        3  593
      1        0        1  594
      1        0        2  594
      1        0        3  594
      1        0        1  595
      1        0        2  595
      1        0        3  595
      1        0        1  596
      1        0        2  596
      1        0        3  596
      1        0        1  597
      1        0        2  597
      1        0        3  597
      1        0        1  598
      1        0        2  598
      1        0        3  598
      1        0        1  599
      1        0        2  599
      1        0        3  599
      1        0        1  600
      1        0        2  600
      1        0        3  600
      1        0        1  601
      1        0        2  601
      1        0        3  601
      1        0        1  602
      1        0        2  602
      1        0        3  602
      1        0        1  603
      1        0        2  603
      1        0        3  603
      1        0        1  604
      1        0        2  604
      1        0        3  604
      1        0        1  605
      1        0        2  605
      1        0        3  605
      1        0        1  606
      1        0        2  606
      1        0        3  606
      1        0        1  607
      1        0        2  607
      1        0        3  607
      1        0        1  608
      1        0        2  608
      1        0        3  608
      1        0        1  609
      1        0        2  609
      1        0        3  609
      1        0        1  610
      1        0        2  610
      1        0        3  610
      1        0        1  611
      1        0        2  611
      1        0        3  611
      1        0        1  612
      1        0        2  612
      1        0        3  612
      1        0        1  613
      1        0        2  613
      1        0        3  613
      1        0        1  614
      1        0        2  614
      1        0        3  614
      1        0        1  615
      1        0        2  615
      1        0        3  615
      1        0        1  616
      1        0        2  616
      1        0        3  616
      1        0        1  617
      1        0        2  617
      1        0        3  617
      1        0        1  618
      1        0        2  618
      1        0        3  618
      1        0        1  619
      1        0        2  619
      1        0        3  619
      1        0        1  620
      1        0        2  620
      1        0        3  620
      1        0        1  621
      1        0        2  621
      1        0        3  621
      1        0        1  622
      1        0        2  622
      1        0        3  622
      1        0        1  623
      1        0        2  623
      1        0        3  623
      1        0        1  624
      1        0        2  624
      1        0        3  624
      1        0        1  625
      1        0        2  625
      1        0        3  625
      1        0        1  626
      1        0        2  626
      1        0        3  626
      1        0        1  627
      1        0        2  627
      1        0        3  627
      1        0        1  628
      1        0        2  628
      1        0        3  628
      1        0        1  629
      1        0        2  629
      1        0        3  629
      1        0        1  630
      1        0        2  630
      1        0        3  630
      1        0        1  631
      1        0        2  631
      1        0        3  631
      1        0        1  632
      1        0        2  632
      1        0        3  632
      1        0        1  633
      1        0        2  633
      1        0        3  633
      1        0        1  634
      1        0        2  634
      1        0        3  634
      1        0        1  635
      1        0        2  635
      1        0        3  635
      1        0        1  636
      1        0        2  636
      1        0        3  636
      1        0        1  637
      1        0        2  637
      1        0        3  637
      1        0        1  638
      1        0        2  638
      1        0        3  638
      1        0        1  639
      1        0        2  639
      1        0        3  639
      1        0        1  640
      1        0        2  640
      1        0        3  640
      1        0        1  641
      1        0        2  641
      1        0        3  641
      1        0        1  642
      1        0        2  642
      1        0        3  642
      1        0        1  643
      1        0        2  643
      1        0        3  643
      1        0        1  644
      1        0        2  644
      1        0        3  644
      1        0        1  645
      1        0        2  645
      1        0        3  645
      1        0        1  646
      1        0        2  646
      1        0        3  646
      1        0        1  647
      1        0        2  647
      1        0        3  647
      1        0        1  648
      1        0        2  648
      1        0        3  648
      1        0        1  649
      1        0        2  649
      1        0        3  649
      1        0        1  650
      1        0        2  650
      1        0        3  650
      1        0        1  651
      1        0        2  651
      1        0        3  651
      1        0        1  652
      1        0        2  652
      1        0        3  652
      1        0        1  653
      1        0        2  653
      1        0        3  653
      1        0        1  654
      1        0        2  654
      1        0        3  654
      1        0        1  655
      1        0        2  655
      1        0        3  655
      1        0        1  656
      1        0        2  656
      1        0        3  656
      1        0        1  657
      1        0        2  657
      1        0        3  657
      1        0        1  658
      1        0        2  658
      1        0        3  658
      1        0        1  659
      1        0        2  659
      1        0        3  659
      1        0        1  660
      1        0        2  660
      1        0        3  660
      1        0        1  661
      1        0        2  661
      1        0        3  661
      1        0        1  662
      1        0        2  662
      1        0        3  662
      1        0        1  663
      1        0        2  663
      1        0        3  663
      1        0        1  664
      1        0        2  664
      1        0        3  664
      1        0        1  665
      1        0        2  665
      1        0        3  665
      1        0        1  666
      1        0        2  666
      1        0        3  666
      1        0        1  667
      1        0        2  667
      1        0        3  667
      1        0        1  668
      1        0        2  668
      1        0        3  668
      1        0        1  669
      1        0        2  669
      1        0        3  669
      1        0        1  670
      1        0        2  670
      1        0        3  670
      1        0        1  671
      1        0        2  671
      1        0        3  671
      1        0        1  672
      1        0        2  672
      1        0        3  672
      1        0        1  673
      1        0        2  673
      1        0        3  673
      1        0        1  674
      1        0        2  674
      1        0        3  674
      1        0        1  675
      1        0        2  675
      1        0        3  675
      1        0        1  676
      1        0        2  676
      1        0        3  676
      1        0        1  677
      1        0        2  677
      1        0        3  677
      1        0        1  678
      1        0        2  678
      1        0        3  678
      1        0        1  679
      1        0        2  679
      1        0        3  679
      1        0        1  680
      1        0        2  680
      1        0        3  680
      1        0        1  681
      1        0        2  681
      1        0        3  681
      1        0        1  682
      1        0        2  682
      1        0        3  682
      1        0        1  683
      1        0        2  683
      1        0        3  683
      1        0        1  684
      1        0        2  684
      1        0        3  684
      1        0        1  685
      1        0        2  685
      1        0        3  685
      1        0        1  686
      1        0        2  686
      1        0        3  686
      1        0        1  687
      1        0        2  687
      1        0        3  687
      1        0        1  688
      1        0        2  688
      1        0        3  688
      1        0        1  689
      1        0        2  689
      1        0        3  689
      1        0        1  690
      1        0        2  690
      1        0        3  690
      1        0        1  691
      1        0        2  691
      1        0        3  691
      1        0        1  692
      1        0        2  692
      1        0        3  692
      1        0        1  693
      1        0        2  693
      1        0        3  693
      1        0        1  694
      1        0        2  694
      1        0        3  694
      1        0        1  695
      1        0        2  695
      1        0        3  695
      1        0        1  696
      1        0        2  696
      1        0        3  696
      1        0        1  697
      1        0        2  697
      1        0        3  697
      1        0        1  698
      1        0        2  698
      1        0        3  698
      1        0        1  699
      1        0        2  699
      1        0        3  699
      1        0        1  700
      1        0        2  700
      1        0        3  700
      1        0        1  701
      1        0        2  701
      1        0        3  701
      1        0        1  702
      1        0        2  702
      1        0        3  702
      1        0        1  703
      1        0        2  703
      1        0        3  703
      1        0        1  704
      1        0        2  704
      1        0        3  704
      1        0        1  705
      1        0        2  705
      1        0        3  705
      1        0        1  706
      1        0        2  706
      1        0        3  706
      1        0        1  707
      1        0        2  707
      1        0        3  707
      1        0        1  708
      1        0        2  708
      1        0        3  708
      1        0        1  709
      1        0        2  709
      1        0        3  709
      1        0        1  710
      1        0        2  710
      1        0        3  710
      1        0        1  711
      1        0        2  711
      1        0        3  711
      1        0        1  712
      1        0        2  712
      1        0        3  712
      1        0        1  713
      1        0        2  713
      1        0        3  713
      1        0        1  714
      1        0        2  714
      1        0        3  714
      1        0        1  715
      1        0        2  715
      1        0        3  715
      1        0        1  716
      1        0        2  716
      1        0        3  716
      1        0        1  717
      1        0        2  717
      1        0        3  717
      1        0        1  718
      1        0        2  718
      1        0        3  718
      1        0        1  719
      1        0        2  719
      1        0        3  719
      1        0        1  720
      1        0        2  720
      1        0        3  720
      1        0        1  721
      1        0        2  721
      1        0        3  721
      1        0        1  722
      1        0        2  722
      1        0        3  722
      1        0        1  723
      1        0        2  723
      1        0        3  723
      1        0        1  724
      1        0        2  724
      1        0        3  724
      1        0        1  725
      1        0        2  725
      1        0        3  725
      1        0        1  726
      1        0        2  726
      1        0        3  726
      1        0        1  727
      1        0        2  727
      1        0        3  727
      1        0        1  728
      1        0        2  728
      1        0        3  728
      1        0        1  729
      1        0        2  729
      1        0        3  729
      1        0        1  730
      1        0        2  730
      1        0        3  730
      1        0        1  731
      1        0        2  731
      1        0        3  731
      1        0        1  732
      1        0        2  732
      1        0        3  732
      1        0        1  733
      1        0        2  733
      1        0        3  733
      1        0        1  734
      1        0        2  734
      1        0        3  734
      1        0        1  735
      1        0        2  735
      1        0        3  735
      1        0        1  736
      1        0        2  736
      1        0        3  736
      1        0        1  737
      1        0        2  737
      1        0        3  737
      1        0        1  738
      1        0        2  738
      1        0        3  738
      1        0        1  739
      1        0        2  739
      1        0        3  739
      1        0        1  740
      1        0        2  740
      1        0        3  740
      1        0        1  741
      1        0        2  741
      1        0        3  741
      1        0        1  742
      1        0        2  742
      1        0        3  742
      1        0        1  743
      1        0        2  743
      1        0        3  743
      1        0        1  744
      1        0        2  744
      1        0        3  744
      1        0        1  745
      1        0        2  745
      1        0        3  745
      1        0        1  746
      1        0        2  746
      1        0        3  746
      1        0        1  747
      1        0        2  747
      1        0        3  747
      1        0        1  748
      1        0        2  748
      1        0        3  748
      1        0        1  749
      1        0        2  749
      1        0        3  749
      1        0        1  750
      1        0        2  750
      1        0        3  750
      1        0        1  751
      1        0        2  751
      1        0        3  751
      1        0        1  752
      1        0        2  752
      1        0        3  752
      1        0        1  753
      1        0        2  753
      1        0        3  753
      1        0        1  754
      1        0        2  754
      1        0        3  754
      1        0        1  755
      1        0        2  755
      1        0        3  755
      1        0        1  756
      1        0        2  756
      1        0        3  756
      1        0        1  757
      1        0        2  757
      1        0        3  757
      1        0        1  758
      1        0        2  758
      1        0        3  758
      1        0        1  759
      1        0        2  759
      1        0        3  759
      1        0        1  760
      1        0        2  760
      1        0        3  760
      1        0        1  761
      1        0        2  761
      1        0        3  761
      1        0        1  762
      1        0        2  762
      1        0        3  762
      1        0        1  763
      1        0        2  763
      1        0        3  763
      1        0        1  764
      1        0        2  764
      1        0        3  764
      1        0        1  765
      1        0        2  765
      1        0        3  765
      1        0        1  766
      1        0        2  766
      1        0        3  766
      1        0        1  767
      1        0        2  767
      1        0        3  767
      1        0        1  768
      1        0        2  768
      1        0        3  768
      1        0        1  769
      1        0        2  769
      1        0        3  769
      1        0        1  770
      1        0        2  770
      1        0        3  770
      1        0        1  771
      1        0        2  771
      1        0        3  771
      1        0        1  772
      1        0        2  772
      1        0        3  772
      1        0        1  773
      1        0        2  773
      1        0        3  773
      1        0        1  774
      1        0        2  774
      1        0        3  774
      1        0        1  775
      1        0        2  775
      1        0        3  775
      1        0        1  776
      1        0        2  776
      1        0        3  776
      1        0        1  777
      1        0        2  777
      1        0        3  777
      1        0        1  778
      1        0        2  778
      1        0        3  778
      1        0        1  779
      1        0        2  779
      1        0        3  779
      1        0        1  780
      1        0        2  780
      1        0        3  780
      1        0        1  781
      1        0        2  781
      1        0        3  781
      1        0        1  782
      1        0        2  782
      1        0        3  782
      1        0        1  783
      1        0        2  783
      1        0        3  783
      1        0        1  784
      1        0        2  784
      1        0        3  784
      1        0        1  785
      1        0        2  785
      1        0        3  785
      1        0        1  786
      1        0        2  786
      1        0        3  786
      1        0        1  787
      1        0        2  787
      1        0        3  787
      1        0        1  788
      1        0        2  788
      1        0        3  788
      1        0        1  789
      1        0        2  789
      1        0        3  789
      1        0        1  790
      1        0        2  790
      1        0        3  790
      1        0        1  791
      1        0        2  791
      1        0        3  791
      1        0        1  792
      1        0        2  792
      1        0        3  792
      1        0        1  793
      1        0        2  793
      1        0        3  793
      1        0        1  794
      1        0        2  794
      1        0        3  794
      1        0        1  795
      1        0        2  795
      1        0        3  795
      1        0        1  796
      1        0        2  796
      1        0        3  796
      1        0        1  797
      1        0        2  797
      1        0        3  797
      1        0        1  798
      1        0        2  798
      1        0        3  798
      1        0        1  799
      1        0        2  799
      1        0        3  799
      1        0        1  800
      1        0        2  800
      1        0        3  800
      1        0        1  801
      1        0        2  801
      1        0        3  801
      1        0        1  802
      1        0        2  802
      1        0        3  802
      1        0        1  803
      1        0        2  803
      1        0        3  803
      1        0        1  804
      1        0        2  804
      1        0        3  804
      1        0        1  805
      1        0        2  805
      1        0        3  805
      1        0        1  806
      1        0        2  806
      1        0        3  806
      1        0        1  807
      1        0        2  807
      1        0        3  807
      1        0        1  808
      1        0        2  808
      1        0        3  808
      1        0        1  809
      1        0        2  809
      1        0        3  809
      1        0        1  810
      1        0        2  810
      1        0        3  810
      1        0        1  811
      1        0        2  811
      1        0        3  811
      1        0        1  812
      1        0        2  812
      1        0        3  812
      1        0        1  813
      1        0        2  813
      1        0        3  813
      1        0        1  814
      1        0        2  814
      1        0        3  814
      1        0        1  815
      1        0        2  815
      1        0        3  815
      1        0        1  816
      1        0        2  816
      1        0        3  816
      1        0        1  817
      1        0        2  817
      1        0        3  817
      1        0        1  818
      1        0        2  818
      1        0        3  818
      1        0        1  819
      1        0        2  819
      1        0        3  819
      1        0        1  820
      1        0        2  820
      1        0        3  820
      1        0        1  821
      1        0        2  821
      1        0        3  821
      1        0        1  822
      1        0        2  822
      1        0        3  822
      1        0        1  823
      1        0        2  823
      1        0        3  823
      1        0        1  824
      1        0        2  824
      1        0        3  824
      1        0        1  825
      1        0        2  825
      1        0        3  825
      1        0        1  826
      1        0        2  826
      1        0        3  826
      1        0        1  827
      1        0        2  827
      1        0        3  827
      1        0        1  828
      1        0        2  828
      1        0        3  828
      1        0        1  829
      1        0        2  829
      1        0        3  829
      1        0        1  830
      1        0        2  830
      1        0        3  830
      1        0        1  831
      1        0        2  831
      1        0        3  831
      1        0        1  832
      1        0        2  832
      1        0        3  832
      1        0        1  833
      1        0        2  833
      1        0        3  833
      1        0        1  834
      1        0        2  834
      1        0        3  834
      1        0        1  835
      1        0        2  835
      1        0        3  835
      1        0        1  836
      1        0        2  836
      1        0        3  836
      1        0        1  837
      1        0        2  837
      1        0        3  837
      1        0        1  838
      1        0        2  838
      1        0        3  838
      1        0        1  839
      1        0        2  839
      1        0        3  839
      1        0        1  840
      1        0        2  840
      1        0        3  840
      1        0        1  841
      1        0        2  841
      1        0        3  841
      1        0        1  842
      1        0        2  842
      1        0        3  842
      1        0        1  843
      1        0        2  843
      1        0        3  843
      1        0        1  844
      1        0        2  844
      1        0        3  844
      1        0        1  845
      1        0        2  845
      1        0        3  845
      1        0        1  846
      1        0        2  846
      1        0        3  846
      1        0        1  847
      1        0        2  847
      1        0        3  847
      1        0        1  848
      1        0        2  848
      1        0        3  848
      1        0        1  849
      1        0        2  849
      1        0        3  849
      1        0        1  850
      1        0        2  850
      1        0        3  850
      1        0        1  851
      1        0        2  851
      1        0        3  851
      1        0        1  852
      1        0        2  852
      1        0        3  852
      1        0        1  853
      1        0        2  853
      1        0        3  853
      1        0        1  854
      1        0        2  854
      1        0        3  854
      1        0        1  855
      1        0        2  855
      1        0        3  855
      1        0        1  856
      1        0        2  856
      1        0        3  856
      1        0        1  857
      1        0        2  857
      1        0        3  857
      1        0        1  858
      1        0        2  858
      1        0        3  858
      1        0        1  859
      1        0        2  859
      1        0        3  859
      1        0        1  860
      1        0        2  860
      1        0        3  860
      1        0        1  861
      1        0        2  861
      1        0        3  861
      1        0        1  862
      1        0        2  862
      1        0        3  862
      1        0        1  863
      1        0        2  863
      1        0        3  863
      1        0        1  864
      1        0        2  864
      1        0        3  864
      1        0        1  865
      1        0        2  865
      1        0        3  865
      1        0        1  866
      1        0        2  866
      1        0        3  866
      1        0        1  867
      1        0        2  867
      1        0        3  867
      1        0        1  868
      1        0        2  868
      1        0        3  868
      1        0        1  869
      1        0        2  869
      1        0        3  869
      1        0        1  870
      1        0        2  870
      1        0        3  870
      1        0        1  871
      1        0        2  871
      1        0        3  871
      1        0        1  872
      1        0        2  872
      1        0        3  872
      1        0        1  873
      1        0        2  873
      1        0        3  873
      1        0        1  874
      1        0        2  874
      1        0        3  874
      1        0        1  875
      1        0        2  875
      1        0        3  875
      1        0        1  876
      1        0        2  876
      1        0        3  876
      1        0        1  877
      1        0        2  877
      1        0        3  877
      1        0        1  878
      1        0        2  878
      1        0        3  878
      1        0        1  879
      1        0        2  879
      1        0        3  879
      1        0        1  880
      1        0        2  880
      1        0        3  880
      1        0        1  881
      1        0        2  881
      1        0        3  881
      1        0        1  882
      1        0        2  882
      1        0        3  882
      1        0        1  883
      1        0        2  883
      1        0        3  883
      1        0        1  884
      1        0        2  884
      1        0        3  884
      1        0        1  885
      1        0        2  885
      1        0        3  885
      1        0        1  886
      1        0        2  886
      1        0        3  886
      1        0        1  887
      1        0        2  887
      1        0        3  887
      1        0        1  888
      1        0        2  888
      1        0        3  888
      1        0        1  889
      1        0        2  889
      1        0        3  889
      1        0        1  890
      1        0        2  890
      1        0        3  890
      1        0        1  891
      1        0        2  891
      1        0        3  891
      1        0        1  892
      1        0        2  892
      1        0        3  892
      1        0        1  893
      1        0        2  893
      1        0        3  893
      1        0        1  894
      1        0        2  894
      1        0        3  894
      1        0        1  895
      1        0        2  895
      1        0        3  895
      1        0        1  896
      1        0        2  896
      1        0        3  896
      1        0        1  897
      1        0        2  897
      1        0        3  897
      1        0        1  898
      1        0        2  898
      1        0        3  898
      1        0        1  899
      1        0        2  899
      1        0        3  899
      1        0        1  900
      1        0        2  900
      1        0        3  900
      1        0        1  901
      1        0        2  901
      1        0        3  901
      1        0        1  902
      1        0        2  902
      1        0        3  902
      1        0        1  903
      1        0        2  903
      1        0        3  903
      1        0        1  904
      1        0        2  904
      1        0        3  904
      1        0        1  905
      1        0        2  905
      1        0        3  905
      1        0        1  906
      1        0        2  906
      1        0        3  906
      1        0        1  907
      1        0        2  907
      1        0        3  907
      1        0        1  908
      1        0        2  908
      1        0        3  908
      1        0        1  909
      1        0        2  909
      1        0        3  909
      1        0        1  910
      1        0        2  910
      1        0        3  910
      1        0        1  911
      1        0        2  911
      1        0        3  911
      1        0        1  912
      1        0        2  912
      1        0        3  912
      1        0        1  913
      1        0        2  913
      1        0        3  913
      1        0        1  914
      1        0        2  914
      1        0        3  914
      1        0        1  915
      1        0        2  915
      1        0        3  915
      1        0        1  916
      1        0        2  916
      1        0        3  916
      1        0        1  917
      1        0        2  917
      1        0        3  917
      1        0        1  918
      1        0        2  918
      1        0        3  918
      1        0        1  919
      1        0        2  919
      1        0        3  919
      1        0        1  920
      1        0        2  920
      1        0        3  920
      1        0        1  921
      1        0        2  921
      1        0        3  921
      1        0        1  922
      1        0        2  922
      1        0        3  922
      1        0        1  923
      1        0        2  923
      1        0        3  923
      1        0        1  924
      1        0        2  924
      1        0        3  924
      1        0        1  925
      1        0        2  925
      1        0        3  925
      1        0        1  926
      1        0        2  926
      1        0        3  926
      1        0        1  927
      1        0        2  927
      1        0        3  927
      1        0        1  928
      1        0        2  928
      1        0        3  928
      1        0        1  929
      1        0        2  929
      1        0        3  929
      1        0        1  930
      1        0        2  930
      1        0        3  930
      1        0        1  931
      1        0        2  931
      1        0        3  931
      1        0        1  932
      1        0        2  932
      1        0        3  932
      1        0        1  933
      1        0        2  933
      1        0        3  933
      1        0        1  934
      1        0        2  934
      1        0        3  934
      1        0        1  935
      1        0        2  935
      1        0        3  935
      1        0        1  936
      1        0        2  936
      1        0        3  936
      1        0        1  937
      1        0        2  937
      1        0        3  937
      1        0        1  938
      1        0        2  938
      1        0        3  938
      1        0        1  939
      1        0        2  939
      1        0        3  939
      1        0        1  940
      1        0        2  940
      1        0        3  940
      1        0        1  941
      1        0        2  941
      1        0        3  941
      1        0        1  942
      1        0        2  942
      1        0        3  942
      1        0        1  943
      1        0        2  943
      1        0        3  943
      1        0        1  944
      1        0        2  944
      1        0        3  944
      1        0        1  945
      1        0        2  945
      1        0        3  945
      1        0        1  946
      1        0        2  946
      1        0        3  946
      1        0        1  947
      1        0        2  947
      1        0        3  947
      1        0        1  948
      1        0        2  948
      1        0        3  948
      1        0        1  949
      1        0        2  949
      1        0        3  949
      1        0        1  950
      1        0        2  950
      1        0        3  950
      1        0        1  951
      1        0        2  951
      1        0        3  951
      1        0        1  952
      1        0        2  952
      1        0        3  952
      1        0        1  953
      1        0        2  953
      1        0        3  953
      1        0        1  954
      1        0        2  954
      1        0        3  954
      1        0        1  955
      1        0        2  955
      1        0        3  955
      1        0        1  956
      1        0        2  956
      1        0        3  956
      1        0        1  957
      1        0        2  957
      1        0        3  957
      1        0        1  958
      1        0        2  958
      1        0        3  958
      1        0        1  959
      1        0        2  959
      1        0        3  959
      1        0        1  960
      1        0        2  960
      1        0        3  960
      1        0        1  961
      1        0        2  961
      1        0        3  961
      1        0        1  962
      1        0        2  962
      1        0        3  962
      1        0        1  963
      1        0        2  963
      1        0        3  963
      1        0        1  964
      1        0        2  964
      1        0        3  964
      1        0        1  965
      1        0        2  965
      1        0        3  965
      1        0        1  966
      1        0        2  966
      1        0        3  966
      1        0        1  967
      1        0        2  967
      1        0        3  967
      1        0        1  968
      1        0        2  968
      1        0        3  968
      1        0        1  969
      1        0        2  969
      1        0        3  969
      1        0        1  970
      1        0        2  970
      1        0        3  970
      1        0        1  971
      1        0        2  971
      1        0        3  971
      1        0        1  972
      1        0        2  972
      1        0        3  972
      1        0        1  973
      1        0        2  973
      1        0        3  973
      1        0        1  974
      1        0        2  974
      1        0        3  974
      1        0        1  975
      1        0        2  975
      1        0        3  975
      1        0        1  976
      1        0        2  976
      1        0        3  976
      1        0        1  977
      1        0        2  977
      1        0        3  977
      1        0        1  978
      1        0        2  978
      1        0        3  978
      1        0        1  979
      1        0        2  979
      1        0        3  979
      1        0        1  980
      1        0        2  980
      1        0        3  980
      1        0        1  981
      1        0        2  981
      1        0        3  981
      1        0        1  982
      1        0        2  982
      1        0        3  982
      1        0        1  983
      1        0        2  983
      1        0        3  983
      1        0        1  984
      1        0        2  984
      1        0        3  984
      1        0        1  985
      1        0        2  985
      1        0        3  985
      1        0        1  986
      1        0        2  986
      1        0        3  986
      1        0        1  987
      1        0        2  987
      1        0        3  987
      1        0        1  988
      1        0        2  988
      1        0        3  988
      1        0        1  989
      1        0        2  989
      1        0        3  989
      1        0        1  990
      1        0        2  990
      1        0        3  990
      1        0        1  991
      1        0        2  991
      1        0        3  991
      1        0        1  992
      1        0        2  992
      1        0        3  992
      1        0        1  993
      1        0        2  993
      1        0        3  993
      1        0        1  994
      1        0        2  994
      1        0        3  994
      1        0        1  995
      1        0        2  995
      1        0        3  995
      1        0        1  996
      1        0        2  996
      1        0        3  996
      1        0        1  997
      1        0        2  997
      1        0        3  997
      1        0        1  998
      1        0        2  998
      1        0        3  998
      1        0        1  999
      1        0        2  999
      1        0        3  999
      1        0        1 1000
      1        0        2 1000
      1        0        3 1000
      1        0        1 1001
      1        0        2 1001
      1        0        3 1001
      1        0        1 1002
      1        0        2 1002
      1        0        3 1002
      1        0        1 1003
      1        0        2 1003
      1        0        3 1003
      1        0        1 1004
      1        0        2 1004
      1        0        3 1004
      1        0        1 1005
      1        0        2 1005
      1        0        3 1005
      1        0        1 1006
      1        0        2 1006
      1        0        3 1006
      1        0        1 1007
      1        0        2 1007
      1        0        3 1007
      1        0        1 1008
      1        0        2 1008
      1        0        3 1008
      1        0        1 1009
      1        0        2 1009
      1        0        3 1009
      1        0        1 1010
      1        0        2 1010
      1        0        3 1010
      1        0        1 1011
      1        0        2 1011
      1        0        3 1011
      1        0        1 1012
      1        0        2 1012
      1        0        3 1012
      1        0        1 1013
      1        0        2 1013
      1        0        3 1013
      1        0        1 1014
      1        0        2 1014
      1        0        3 1014
      1        0        1 1015
      1        0        2 1015
      1        0        3 1015
      1        0        1 1016
      1        0        2 1016
      1        0        3 1016
      1        0        1 1017
      1        0        2 1017
      1        0        3 1017
      1        0        1 1018
      1        0        2 1018
      1        0        3 1018
      1        0        1 1019
      1        0        2 1019
      1        0        3 1019
      1        0        1 1020
      1        0        2 1020
      1        0        3 1020
      1        0        1 1021
      1        0        2 1021
      1        0        3 1021
      1        0        1 1022
      1        0        2 1022
      1        0        3 1022
      1        0        1 1023
      1        0        2 1023
      1        0        3 1023
      1        0        1 1024
      1        0        2 1024
      1        0        3 1024
      1        0        1 1025
      1        0        2 1025
      1        0        3 1025
      1        0        1 1026
      1        0        2 1026
      1        0        3 1026
      1        0        1 1027
      1        0        2 1027
      1        0        3 1027
      1        0        1 1028
      1        0        2 1028
      1        0        3 1028
      1        0        1 1029
      1        0        2 1029
      1        0        3 1029
      1        0        1 1030
      1        0        2 1030
      1        0        3 1030
      1        0        1 1031
      1        0        2 1031
      1        0        3 1031
      1        0        1 1032
      1        0        2 1032
      1        0        3 1032
      1        0        1 1033
      1        0        2 1033
      1        0        3 1033
      1        0        1 1034
      1        0        2 1034
      1        0        3 1034
      1        0        1 1035
      1        0        2 1035
      1        0        3 1035
      1        0        1 1036
      1        0        2 1036
      1        0        3 1036
      1        0        1 1037
      1        0        2 1037
      1        0        3 1037
      0        1        1 1038
      0        1        2 1038
      0        1        3 1038
      0        1        1 1039
      0        1        2 1039
      0        1        3 1039
      0        1        1 1040
      0        1        2 1040
      0        1        3 1040
      0        1        1 1041
      0        1        2 1041
      0        1        3 1041
      0        1        1 1042
      0        1        2 1042
      0        1        3 1042
      0        1        1 1043
      0        1        2 1043
      0        1        3 1043
      0        1        1 1044
      0        1        2 1044
      0        1        3 1044
      0        1        1 1045
      0        1        2 1045
      0        1        3 1045
      0        1        1 1046
      0        1        2 1046
      0        1        3 1046
      0        1        1 1047
      0        1        2 1047
      0        1        3 1047
      0        1        1 1048
      0        1        2 1048
      0        1        3 1048
      0        1        1 1049
      0        1        2 1049
      0        1        3 1049
      0        1        1 1050
      0        1        2 1050
      0        1        3 1050
      0        1        1 1051
      0        1        2 1051
      0        1        3 1051
      0        1        1 1052
      0        1        2 1052
      0        1        3 1052
      0        1        1 1053
      0        1        2 1053
      0        1        3 1053
      0        1        1 1054
      0        1        2 1054
      0        1        3 1054
      0        1        1 1055
      0        1        2 1055
      0        1        3 1055
      0        1        1 1056
      0        1        2 1056
      0        1        3 1056
      0        1        1 1057
      0        1        2 1057
      0        1        3 1057
      0        1        1 1058
      0        1        2 1058
      0        1        3 1058
      0        1        1 1059
      0        1        2 1059
      0        1        3 1059
      0        1        1 1060
      0        1        2 1060
      0        1        3 1060
      0        1        1 1061
      0        1        2 1061
      0        1        3 1061
      0        1        1 1062
      0        1        2 1062
      0        1        3 1062
      0        1        1 1063
      0        1        2 1063
      0        1        3 1063
      0        1        1 1064
      0        1        2 1064
      0        1        3 1064
      0        1        1 1065
      0        1        2 1065
      0        1        3 1065
      0        1        1 1066
      0        1        2 1066
      0        1        3 1066
      0        1        1 1067
      0        1        2 1067
      0        1        3 1067
      0        1        1 1068
      0        1        2 1068
      0        1        3 1068
      0        1        1 1069
      0        1        2 1069
      0        1        3 1069
      0        1        1 1070
      0        1        2 1070
      0        1        3 1070
      0        1        1 1071
      0        1        2 1071
      0        1        3 1071
      0        1        1 1072
      0        1        2 1072
      0        1        3 1072
      0        1        1 1073
      0        1        2 1073
      0        1        3 1073
      0        1        1 1074
      0        1        2 1074
      0        1        3 1074
      0        1        1 1075
      0        1        2 1075
      0        1        3 1075
      0        1        1 1076
      0        1        2 1076
      0        1        3 1076
      0        1        1 1077
      0        1        2 1077
      0        1        3 1077
      0        1        1 1078
      0        1        2 1078
      0        1        3 1078
      0        1        1 1079
      0        1        2 1079
      0        1        3 1079
      0        1        1 1080
      0        1        2 1080
      0        1        3 1080
      0        1        1 1081
      0        1        2 1081
      0        1        3 1081
      0        1        1 1082
      0        1        2 1082
      0        1        3 1082
      0        1        1 1083
      0        1        2 1083
      0        1        3 1083
      0        1        1 1084
      0        1        2 1084
      0        1        3 1084
      0        1        1 1085
      0        1        2 1085
      0        1        3 1085
      0        1        1 1086
      0        1        2 1086
      0        1        3 1086
      0        1        1 1087
      0        1        2 1087
      0        1        3 1087
      0        1        1 1088
      0        1        2 1088
      0        1        3 1088
      0        1        1 1089
      0        1        2 1089
      0        1        3 1089
      0        1        1 1090
      0        1        2 1090
      0        1        3 1090
      0        1        1 1091
      0        1        2 1091
      0        1        3 1091
      0        1        1 1092
      0        1        2 1092
      0        1        3 1092
      0        1        1 1093
      0        1        2 1093
      0        1        3 1093
      0        1        1 1094
      0        1        2 1094
      0        1        3 1094
      0        1        1 1095
      0        1        2 1095
      0        1        3 1095
      0        1        1 1096
      0        1        2 1096
      0        1        3 1096
      0        1        1 1097
      0        1        2 1097
      0        1        3 1097
      0        1        1 1098
      0        1        2 1098
      0        1        3 1098
      0        1        1 1099
      0        1        2 1099
      0        1        3 1099
      0        1        1 1100
      0        1        2 1100
      0        1        3 1100
      0        1        1 1101
      0        1        2 1101
      0        1        3 1101
      0        1        1 1102
      0        1        2 1102
      0        1        3 1102
      0        1        1 1103
      0        1        2 1103
      0        1        3 1103
      0        1        1 1104
      0        1        2 1104
      0        1        3 1104
      0        1        1 1105
      0        1        2 1105
      0        1        3 1105
      0        1        1 1106
      0        1        2 1106
      0        1        3 1106
      0        1        1 1107
      0        1        2 1107
      0        1        3 1107
      0        1        1 1108
      0        1        2 1108
      0        1        3 1108
      0        1        1 1109
      0        1        2 1109
      0        1        3 1109
      0        1        1 1110
      0        1        2 1110
      0        1        3 1110
      0        1        1 1111
      0        1        2 1111
      0        1        3 1111
      0        1        1 1112
      0        1        2 1112
      0        1        3 1112
      0        1        1 1113
      0        1        2 1113
      0        1        3 1113
      0        1        1 1114
      0        1        2 1114
      0        1        3 1114
      0        1        1 1115
      0        1        2 1115
      0        1        3 1115
      0        1        1 1116
      0        1        2 1116
      0        1        3 1116
      0        1        1 1117
      0        1        2 1117
      0        1        3 1117
      0        1        1 1118
      0        1        2 1118
      0        1        3 1118
      0        1        1 1119
      0        1        2 1119
      0        1        3 1119
      0        1        1 1120
      0        1        2 1120
      0        1        3 1120
      0        1        1 1121
      0        1        2 1121
      0        1        3 1121
      0        1        1 1122
      0        1        2 1122
      0        1        3 1122
      0        1        1 1123
      0        1        2 1123
      0        1        3 1123
      0        1        1 1124
      0        1        2 1124
      0        1        3 1124
      0        1        1 1125
      0        1        2 1125
      0        1        3 1125
      0        1        1 1126
      0        1        2 1126
      0        1        3 1126
      0        1        1 1127
      0        1        2 1127
      0        1        3 1127
      0        1        1 1128
      0        1        2 1128
      0        1        3 1128
      0        1        1 1129
      0        1        2 1129
      0        1        3 1129
      0        1        1 1130
      0        1        2 1130
      0        1        3 1130
      0        1        1 1131
      0        1        2 1131
      0        1        3 1131
      0        1        1 1132
      0        1        2 1132
      0        1        3 1132
      0        1        1 1133
      0        1        2 1133
      0        1        3 1133
      0        1        1 1134
      0        1        2 1134
      0        1        3 1134
      0        1        1 1135
      0        1        2 1135
      0        1        3 1135
      0        1        1 1136
      0        1        2 1136
      0        1        3 1136
      0        1        1 1137
      0        1        2 1137
      0        1        3 1137
      0        1        1 1138
      0        1        2 1138
      0        1        3 1138
      0        1        1 1139
      0        1        2 1139
      0        1        3 1139
      0        1        1 1140
      0        1        2 1140
      0        1        3 1140
      0        1        1 1141
      0        1        2 1141
      0        1        3 1141
      0        1        1 1142
      0        1        2 1142
      0        1        3 1142
      0        1        1 1143
      0        1        2 1143
      0        1        3 1143
      0        1        1 1144
      0        1        2 1144
      0        1        3 1144
      0        1        1 1145
      0        1        2 1145
      0        1        3 1145
      0        1        1 1146
      0        1        2 1146
      0        1        3 1146
      0        1        1 1147
      0        1        2 1147
      0        1        3 1147
      0        1        1 1148
      0        1        2 1148
      0        1        3 1148
      0        1        1 1149
      0        1        2 1149
      0        1        3 1149
      0        1        1 1150
      0        1        2 1150
      0        1        3 1150
      0        1        1 1151
      0        1        2 1151
      0        1        3 1151
      0        1        1 1152
      0        1        2 1152
      0        1        3 1152
      0        1        1 1153
      0        1        2 1153
      0        1        3 1153
      0        1        1 1154
      0        1        2 1154
      0        1        3 1154
      0        1        1 1155
      0        1        2 1155
      0        1        3 1155
      0        1        1 1156
      0        1        2 1156
      0        1        3 1156
      0        1        1 1157
      0        1        2 1157
      0        1        3 1157
      0        1        1 1158
      0        1        2 1158
      0        1        3 1158
      0        1        1 1159
      0        1        2 1159
      0        1        3 1159
      0        1        1 1160
      0        1        2 1160
      0        1        3 1160
      0        1        1 1161
      0        1        2 1161
      0        1        3 1161
      0        1        1 1162
      0        1        2 1162
      0        1        3 1162
      0        1        1 1163
      0        1        2 1163
      0        1        3 1163
      0        1        1 1164
      0        1        2 1164
      0        1        3 1164
      0        1        1 1165
      0        1        2 1165
      0        1        3 1165
      0        1        1 1166
      0        1        2 1166
      0        1        3 1166
      0        1        1 1167
      0        1        2 1167
      0        1        3 1167
      0        1        1 1168
      0        1        2 1168
      0        1        3 1168
      0        1        1 1169
      0        1        2 1169
      0        1        3 1169
      0        1        1 1170
      0        1        2 1170
      0        1        3 1170
      0        1        1 1171
      0        1        2 1171
      0        1        3 1171
      0        1        1 1172
      0        1        2 1172
      0        1        3 1172
      0        1        1 1173
      0        1        2 1173
      0        1        3 1173
      0        1        1 1174
      0        1        2 1174
      0        1        3 1174
      0        1        1 1175
      0        1        2 1175
      0        1        3 1175
      0        1        1 1176
      0        1        2 1176
      0        1        3 1176
      0        1        1 1177
      0        1        2 1177
      0        1        3 1177
      0        1        1 1178
      0        1        2 1178
      0        1        3 1178
      0        1        1 1179
      0        1        2 1179
      0        1        3 1179
      0        1        1 1180
      0        1        2 1180
      0        1        3 1180
      0        1        1 1181
      0        1        2 1181
      0        1        3 1181
      0        1        1 1182
      0        1        2 1182
      0        1        3 1182
      0        1        1 1183
      0        1        2 1183
      0        1        3 1183
      0        1        1 1184
      0        1        2 1184
      0        1        3 1184
      0        1        1 1185
      0        1        2 1185
      0        1        3 1185
      0        1        1 1186
      0        1        2 1186
      0        1        3 1186
      0        1        1 1187
      0        1        2 1187
      0        1        3 1187
      0        1        1 1188
      0        1        2 1188
      0        1        3 1188
      0        1        1 1189
      0        1        2 1189
      0        1        3 1189
      0        1        1 1190
      0        1        2 1190
      0        1        3 1190
      0        1        1 1191
      0        1        2 1191
      0        1        3 1191
      0        1        1 1192
      0        1        2 1192
      0        1        3 1192
      0        1        1 1193
      0        1        2 1193
      0        1        3 1193
      0        1        1 1194
      0        1        2 1194
      0        1        3 1194
      0        1        1 1195
      0        1        2 1195
      0        1        3 1195
      0        1        1 1196
      0        1        2 1196
      0        1        3 1196
      0        1        1 1197
      0        1        2 1197
      0        1        3 1197
      0        1        1 1198
      0        1        2 1198
      0        1        3 1198
      0        1        1 1199
      0        1        2 1199
      0        1        3 1199
      0        1        1 1200
      0        1        2 1200
      0        1        3 1200
      0        1        1 1201
      0        1        2 1201
      0        1        3 1201
      0        1        1 1202
      0        1        2 1202
      0        1        3 1202
      0        1        1 1203
      0        1        2 1203
      0        1        3 1203
      0        1        1 1204
      0        1        2 1204
      0        1        3 1204
      0        1        1 1205
      0        1        2 1205
      0        1        3 1205
      0        1        1 1206
      0        1        2 1206
      0        1        3 1206
      0        1        1 1207
      0        1        2 1207
      0        1        3 1207
      0        1        1 1208
      0        1        2 1208
      0        1        3 1208
      0        1        1 1209
      0        1        2 1209
      0        1        3 1209
      0        1        1 1210
      0        1        2 1210
      0        1        3 1210
      0        1        1 1211
      0        1        2 1211
      0        1        3 1211
      0        1        1 1212
      0        1        2 1212
      0        1        3 1212
      0        1        1 1213
      0        1        2 1213
      0        1        3 1213
      0        1        1 1214
      0        1        2 1214
      0        1        3 1214
      0        1        1 1215
      0        1        2 1215
      0        1        3 1215
      0        1        1 1216
      0        1        2 1216
      0        1        3 1216
      0        1        1 1217
      0        1        2 1217
      0        1        3 1217
      0        1        1 1218
      0        1        2 1218
      0        1        3 1218
      0        1        1 1219
      0        1        2 1219
      0        1        3 1219
      0        1        1 1220
      0        1        2 1220
      0        1        3 1220
      0        1        1 1221
      0        1        2 1221
      0        1        3 1221
      0        1        1 1222
      0        1        2 1222
      0        1        3 1222
      0        1        1 1223
      0        1        2 1223
      0        1        3 1223
      0        1        1 1224
      0        1        2 1224
      0        1        3 1224
      0        1        1 1225
      0        1        2 1225
      0        1        3 1225
      0        1        1 1226
      0        1        2 1226
      0        1        3 1226
      0        1        1 1227
      0        1        2 1227
      0        1        3 1227
      0        1        1 1228
      0        1        2 1228
      0        1        3 1228
      0        1        1 1229
      0        1        2 1229
      0        1        3 1229
      0        1        1 1230
      0        1        2 1230
      0        1        3 1230
      0        1        1 1231
      0        1        2 1231
      0        1        3 1231
      0        1        1 1232
      0        1        2 1232
      0        1        3 1232
      0        1        1 1233
      0        1        2 1233
      0        1        3 1233
      0        1        1 1234
      0        1        2 1234
      0        1        3 1234
      0        1        1 1235
      0        1        2 1235
      0        1        3 1235
      0        1        1 1236
      0        1        2 1236
      0        1        3 1236
      0        1        1 1237
      0        1        2 1237
      0        1        3 1237
      0        1        1 1238
      0        1        2 1238
      0        1        3 1238
      0        1        1 1239
      0        1        2 1239
      0        1        3 1239
      0        1        1 1240
      0        1        2 1240
      0        1        3 1240
      0        1        1 1241
      0        1        2 1241
      0        1        3 1241
      0        1        1 1242
      0        1        2 1242
      0        1        3 1242
      0        1        1 1243
      0        1        2 1243
      0        1        3 1243
      0        1        1 1244
      0        1        2 1244
      0        1        3 1244
      0        1        1 1245
      0        1        2 1245
      0        1        3 1245
      0        1        1 1246
      0        1        2 1246
      0        1        3 1246
      0        1        1 1247
      0        1        2 1247
      0        1        3 1247
      0        1        1 1248
      0        1        2 1248
      0        1        3 1248
      0        1        1 1249
      0        1        2 1249
      0        1        3 1249
      0        1        1 1250
      0        1        2 1250
      0        1        3 1250
      0        1        1 1251
      0        1        2 1251
      0        1        3 1251
      0        1        1 1252
      0        1        2 1252
      0        1        3 1252
      0        1        1 1253
      0        1        2 1253
      0        1        3 1253
      0        1        1 1254
      0        1        2 1254
      0        1        3 1254
      0        1        1 1255
      0        1        2 1255
      0        1        3 1255
      0        1        1 1256
      0        1        2 1256
      0        1        3 1256
      0        1        1 1257
      0        1        2 1257
      0        1        3 1257
      0        1        1 1258
      0        1        2 1258
      0        1        3 1258
      0        1        1 1259
      0        1        2 1259
      0        1        3 1259
      0        1        1 1260
      0        1        2 1260
      0        1        3 1260
      0        1        1 1261
      0        1        2 1261
      0        1        3 1261
      0        1        1 1262
      0        1        2 1262
      0        1        3 1262
      0        1        1 1263
      0        1        2 1263
      0        1        3 1263
      0        1        1 1264
      0        1        2 1264
      0        1        3 1264
      0        1        1 1265
      0        1        2 1265
      0        1        3 1265
      0        1        1 1266
      0        1        2 1266
      0        1        3 1266
      0        1        1 1267
      0        1        2 1267
      0        1        3 1267
      0        1        1 1268
      0        1        2 1268
      0        1        3 1268
      0        1        1 1269
      0        1        2 1269
      0        1        3 1269
      0        1        1 1270
      0        1        2 1270
      0        1        3 1270
      0        1        1 1271
      0        1        2 1271
      0        1        3 1271
      0        1        1 1272
      0        1        2 1272
      0        1        3 1272
      0        1        1 1273
      0        1        2 1273
      0        1        3 1273
      0        1        1 1274
      0        1        2 1274
      0        1        3 1274
      0        1        1 1275
      0        1        2 1275
      0        1        3 1275
      0        1        1 1276
      0        1        2 1276
      0        1        3 1276
      0        1        1 1277
      0        1        2 1277
      0        1        3 1277
      0        1        1 1278
      0        1        2 1278
      0        1        3 1278
      0        1        1 1279
      0        1        2 1279
      0        1        3 1279
      0        1        1 1280
      0        1        2 1280
      0        1        3 1280
      0        1        1 1281
      0        1        2 1281
      0        1        3 1281
      0        1        1 1282
      0        1        2 1282
      0        1        3 1282
      0        1        1 1283
      0        1        2 1283
      0        1        3 1283
      0        1        1 1284
      0        1        2 1284
      0        1        3 1284
      0        1        1 1285
      0        1        2 1285
      0        1        3 1285
      0        1        1 1286
      0        1        2 1286
      0        1        3 1286
      0        1        1 1287
      0        1        2 1287
      0        1        3 1287
      0        1        1 1288
      0        1        2 1288
      0        1        3 1288
      0        1        1 1289
      0        1        2 1289
      0        1        3 1289
      0        1        1 1290
      0        1        2 1290
      0        1        3 1290
      0        1        1 1291
      0        1        2 1291
      0        1        3 1291
      0        1        1 1292
      0        1        2 1292
      0        1        3 1292
      0        1        1 1293
      0        1        2 1293
      0        1        3 1293
      0        1        1 1294
      0        1        2 1294
      0        1        3 1294
      0        1        1 1295
      0        1        2 1295
      0        1        3 1295
      0        1        1 1296
      0        1        2 1296
      0        1        3 1296
      0        1        1 1297
      0        1        2 1297
      0        1        3 1297
      0        1        1 1298
      0        1        2 1298
      0        1        3 1298
      0        1        1 1299
      0        1        2 1299
      0        1        3 1299
      0        1        1 1300
      0        1        2 1300
      0        1        3 1300
      0        1        1 1301
      0        1        2 1301
      0        1        3 1301
      0        1        1 1302
      0        1        2 1302
      0        1        3 1302
      0        1        1 1303
      0        1        2 1303
      0        1        3 1303
      0        1        1 1304
      0        1        2 1304
      0        1        3 1304
      0        1        1 1305
      0        1        2 1305
      0        1        3 1305
      0        1        1 1306
      0        1        2 1306
      0        1        3 1306
      0        1        1 1307
      0        1        2 1307
      0        1        3 1307
      0        1        1 1308
      0        1        2 1308
      0        1        3 1308
      0        1        1 1309
      0        1        2 1309
      0        1        3 1309
      0        1        1 1310
      0        1        2 1310
      0        1        3 1310
      0        1        1 1311
      0        1        2 1311
      0        1        3 1311
      0        1        1 1312
      0        1        2 1312
      0        1        3 1312
      0        1        1 1313
      0        1        2 1313
      0        1        3 1313
      0        1        1 1314
      0        1        2 1314
      0        1        3 1314
      0        1        1 1315
      0        1        2 1315
      0        1        3 1315
      0        1        1 1316
      0        1        2 1316
      0        1        3 1316
      0        1        1 1317
      0        1        2 1317
      0        1        3 1317
      0        1        1 1318
      0        1        2 1318
      0        1        3 1318
      0        1        1 1319
      0        1        2 1319
      0        1        3 1319
      0        1        1 1320
      0        1        2 1320
      0        1        3 1320
      0        1        1 1321
      0        1        2 1321
      0        1        3 1321
      0        1        1 1322
      0        1        2 1322
      0        1        3 1322
      0        1        1 1323
      0        1        2 1323
      0        1        3 1323
      0        1        1 1324
      0        1        2 1324
      0        1        3 1324
      0        1        1 1325
      0        1        2 1325
      0        1        3 1325
      0        1        1 1326
      0        1        2 1326
      0        1        3 1326
      0        1        1 1327
      0        1        2 1327
      0        1        3 1327
      0        1        1 1328
      0        1        2 1328
      0        1        3 1328
      0        1        1 1329
      0        1        2 1329
      0        1        3 1329
      0        1        1 1330
      0        1        2 1330
      0        1        3 1330
      0        1        1 1331
      0        1        2 1331
      0        1        3 1331
      0        1        1 1332
      0        1        2 1332
      0        1        3 1332
      0        1        1 1333
      0        1        2 1333
      0        1        3 1333
      0        1        1 1334
      0        1        2 1334
      0        1        3 1334
      0        1        1 1335
      0        1        2 1335
      0        1        3 1335
      0        1        1 1336
      0        1        2 1336
      0        1        3 1336
      0        1        1 1337
      0        1        2 1337
      0        1        3 1337
      0        1        1 1338
      0        1        2 1338
      0        1        3 1338
      0        1        1 1339
      0        1        2 1339
      0        1        3 1339
      0        1        1 1340
      0        1        2 1340
      0        1        3 1340
      0        1        1 1341
      0        1        2 1341
      0        1        3 1341
      0        1        1 1342
      0        1        2 1342
      0        1        3 1342
      0        1        1 1343
      0        1        2 1343
      0        1        3 1343
      0        1        1 1344
      0        1        2 1344
      0        1        3 1344
      0        1        1 1345
      0        1        2 1345
      0        1        3 1345
      0        1        1 1346
      0        1        2 1346
      0        1        3 1346
      0        1        1 1347
      0        1        2 1347
      0        1        3 1347
      0        1        1 1348
      0        1        2 1348
      0        1        3 1348
      0        1        1 1349
      0        1        2 1349
      0        1        3 1349
      0        1        1 1350
      0        1        2 1350
      0        1        3 1350
      0        1        1 1351
      0        1        2 1351
      0        1        3 1351
      0        1        1 1352
      0        1        2 1352
      0        1        3 1352
      0        1        1 1353
      0        1        2 1353
      0        1        3 1353
      0        1        1 1354
      0        1        2 1354
      0        1        3 1354
      0        1        1 1355
      0        1        2 1355
      0        1        3 1355
      0        1        1 1356
      0        1        2 1356
      0        1        3 1356
      0        1        1 1357
      0        1        2 1357
      0        1        3 1357
      0        1        1 1358
      0        1        2 1358
      0        1        3 1358
      0        1        1 1359
      0        1        2 1359
      0        1        3 1359
      0        1        1 1360
      0        1        2 1360
      0        1        3 1360
      0        1        1 1361
      0        1        2 1361
      0        1        3 1361
      0        1        1 1362
      0        1        2 1362
      0        1        3 1362
      0        1        1 1363
      0        1        2 1363
      0        1        3 1363
      0        1        1 1364
      0        1        2 1364
      0        1        3 1364
      0        1        1 1365
      0        1        2 1365
      0        1        3 1365
      0        1        1 1366
      0        1        2 1366
      0        1        3 1366
      0        1        1 1367
      0        1        2 1367
      0        1        3 1367
      0        1        1 1368
      0        1        2 1368
      0        1        3 1368
      0        1        1 1369
      0        1        2 1369
      0        1        3 1369
      0        1        1 1370
      0        1        2 1370
      0        1        3 1370
      0        1        1 1371
      0        1        2 1371
      0        1        3 1371
      0        1        1 1372
      0        1        2 1372
      0        1        3 1372
      0        1        1 1373
      0        1        2 1373
      0        1        3 1373
      0        1        1 1374
      0        1        2 1374
      0        1        3 1374
      0        1        1 1375
      0        1        2 1375
      0        1        3 1375
      0        1        1 1376
      0        1        2 1376
      0        1        3 1376
      0        1        1 1377
      0        1        2 1377
      0        1        3 1377
      0        1        1 1378
      0        1        2 1378
      0        1        3 1378
      0        1        1 1379
      0        1        2 1379
      0        1        3 1379
      0        1        1 1380
      0        1        2 1380
      0        0        3 1380
      0        1        1 1381
      0        1        2 1381
      0        0        3 1381
      0        1        1 1382
      0        1        2 1382
      0        0        3 1382
      0        1        1 1383
      0        1        2 1383
      0        0        3 1383
      0        1        1 1384
      0        1        2 1384
      0        0        3 1384
      0        1        1 1385
      0        1        2 1385
      0        0        3 1385
      0        1        1 1386
      0        1        2 1386
      0        0        3 1386
      0        1        1 1387
      0        1        2 1387
      0        0        3 1387
      0        1        1 1388
      0        1        2 1388
      0        0        3 1388
      0        1        1 1389
      0        1        2 1389
      0        0        3 1389
      0        1        1 1390
      0        1        2 1390
      0        0        3 1390
      0        1        1 1391
      0        1        2 1391
      0        0        3 1391
      0        1        1 1392
      0        1        2 1392
      0        0        3 1392
      0        1        1 1393
      0        1        2 1393
      0        0        3 1393
      0        1        1 1394
      0        1        2 1394
      0        0        3 1394
      0        1        1 1395
      0        1        2 1395
      0        0        3 1395
      0        1        1 1396
      0        1        2 1396
      0        0        3 1396
      0        1        1 1397
      0        1        2 1397
      0        0        3 1397
      0        1        1 1398
      0        1        2 1398
      0        0        3 1398
      0        1        1 1399
      0        1        2 1399
      0        0        3 1399
      0        1        1 1400
      0        1        2 1400
      0        0        3 1400
      0        1        1 1401
      0        1        2 1401
      0        0        3 1401
      0        1        1 1402
      0        1        2 1402
      0        0        3 1402
      0        1        1 1403
      0        1        2 1403
      0        0        3 1403
      0        1        1 1404
      0        1        2 1404
      0        0        3 1404
      0        1        1 1405
      0        1        2 1405
      0        0        3 1405
      0        0        1 1406
      0        1        2 1406
      0        1        3 1406
      0        0        1 1407
      0        1        2 1407
      0        1        3 1407
      0        0        1 1408
      0        1        2 1408
      0        1        3 1408
      0        0        1 1409
      0        1        2 1409
      0        1        3 1409
      0        0        1 1410
      0        1        2 1410
      0        1        3 1410
      0        0        1 1411
      0        1        2 1411
      0        1        3 1411
      0        0        1 1412
      0        1        2 1412
      0        0        3 1412
      0        0        1 1413
      0        1        2 1413
      0        0        3 1413
      0        0        1 1414
      0        1        2 1414
      0        0        3 1414
      0        0        1 1415
      0        1        2 1415
      0        0        3 1415
      0        0        1 1416
      0        1        2 1416
      0        0        3 1416
      0        0        1 1417
      0        1        2 1417
      0        0        3 1417
      0        0        1 1418
      0        1        2 1418
      0        0        3 1418
      0        0        1 1419
      0        1        2 1419
      0        0        3 1419
      0        0        1 1420
      0        1        2 1420
      0        0        3 1420
      0        0        1 1421
      0        1        2 1421
      0        0        3 1421
      0        0        1 1422
      0        1        2 1422
      0        0        3 1422
      0        0        1 1423
      0        1        2 1423
      0        0        3 1423
      0        0        1 1424
      0        1        2 1424
      0        0        3 1424
      0        0        1 1425
      0        1        2 1425
      0        0        3 1425
      0        0        1 1426
      0        1        2 1426
      0        0        3 1426
      0        0        1 1427
      0        1        2 1427
      0        0        3 1427
      0        0        1 1428
      0        1        2 1428
      0        0        3 1428
      0        0        1 1429
      0        1        2 1429
      0        0        3 1429
      0        0        1 1430
      0        1        2 1430
      0        0        3 1430
      0        0        1 1431
      0        1        2 1431
      0        0        3 1431
      0        0        1 1432
      0        1        2 1432
      0        0        3 1432
      0        1        1 1433
      0        0        2 1433
      0        1        3 1433
      0        1        1 1434
      0        0        2 1434
      0        1        3 1434
      0        1        1 1435
      0        0        2 1435
      0        1        3 1435
      0        1        1 1436
      0        0        2 1436
      0        1        3 1436
      0        1        1 1437
      0        0        2 1437
      0        1        3 1437
      0        1        1 1438
      0        0        2 1438
      0        1        3 1438
      0        1        1 1439
      0        0        2 1439
      0        1        3 1439
      0        1        1 1440
      0        0        2 1440
      0        1        3 1440
      0        1        1 1441
      0        0        2 1441
      0        1        3 1441
      0        1        1 1442
      0        0        2 1442
      0        1        3 1442
      0        1        1 1443
      0        0        2 1443
      0        1        3 1443
      0        1        1 1444
      0        0        2 1444
      0        0        3 1444
      0        1        1 1445
      0        0        2 1445
      0        0        3 1445
      0        1        1 1446
      0        0        2 1446
      0        0        3 1446
      0        1        1 1447
      0        0        2 1447
      0        0        3 1447
      0        1        1 1448
      0        0        2 1448
      0        0        3 1448
      0        1        1 1449
      0        0        2 1449
      0        0        3 1449
      0        1        1 1450
      0        0        2 1450
      0        0        3 1450
      0        1        1 1451
      0        0        2 1451
      0        0        3 1451
      0        1        1 1452
      0        0        2 1452
      0        0        3 1452
      0        1        1 1453
      0        0        2 1453
      0        0        3 1453
      0        1        1 1454
      0        0        2 1454
      0        0        3 1454
      0        1        1 1455
      0        0        2 1455
      0        0        3 1455
      0        1        1 1456
      0        0        2 1456
      0        0        3 1456
      0        1        1 1457
      0        0        2 1457
      0        0        3 1457
      0        1        1 1458
      0        0        2 1458
      0        0        3 1458
      0        1        1 1459
      0        0        2 1459
      0        0        3 1459
      0        1        1 1460
      0        0        2 1460
      0        0        3 1460
      0        1        1 1461
      0        0        2 1461
      0        0        3 1461
      0        1        1 1462
      0        0        2 1462
      0        0        3 1462
      0        1        1 1463
      0        0        2 1463
      0        0        3 1463
      0        1        1 1464
      0        0        2 1464
      0        0        3 1464
      0        1        1 1465
      0        0        2 1465
      0        0        3 1465
      0        1        1 1466
      0        0        2 1466
      0        0        3 1466
      0        1        1 1467
      0        0        2 1467
      0        0        3 1467
      0        1        1 1468
      0        0        2 1468
      0        0        3 1468
      0        1        1 1469
      0        0        2 1469
      0        0        3 1469
      0        1        1 1470
      0        0        2 1470
      0        0        3 1470
      0        1        1 1471
      0        0        2 1471
      0        0        3 1471
      0        1        1 1472
      0        0        2 1472
      0        0        3 1472
      0        1        1 1473
      0        0        2 1473
      0        0        3 1473
      0        1        1 1474
      0        0        2 1474
      0        0        3 1474
      0        1        1 1475
      0        0        2 1475
      0        0        3 1475
      0        0        1 1476
      0        0        2 1476
      0        1        3 1476
      0        0        1 1477
      0        0        2 1477
      0        1        3 1477
      0        0        1 1478
      0        0        2 1478
      0        1        3 1478
      0        0        1 1479
      0        0        2 1479
      0        1        3 1479
      0        0        1 1480
      0        0        2 1480
      0        1        3 1480
      0        0        1 1481
      0        0        2 1481
      0        1        3 1481
      0        0        1 1482
      0        0        2 1482
      0        1        3 1482
      0        0        1 1483
      0        0        2 1483
      0        1        3 1483
      0        0        1 1484
      0        0        2 1484
      0        1        3 1484
      0        0        1 1485
      0        0        2 1485
      0        1        3 1485
      0        0        1 1486
      0        0        2 1486
      0        1        3 1486
      0        0        1 1487
      0        0        2 1487
      0        1        3 1487
      0        0        1 1488
      0        0        2 1488
      0        1        3 1488
      0        0        1 1489
      0        0        2 1489
      0        1        3 1489
      0        0        1 1490
      0        0        2 1490
      0        1        3 1490
      0        0        1 1491
      0        0        2 1491
      0        1        3 1491
      0        0        1 1492
      0        0        2 1492
      0        1        3 1492
      0        0        1 1493
      0        0        2 1493
      0        1        3 1493
      0        0        1 1494
      0        0        2 1494
      0        1        3 1494
      0        0        1 1495
      0        0        2 1495
      0        0        3 1495
      0        0        1 1496
      0        0        2 1496
      0        0        3 1496
      0        0        1 1497
      0        0        2 1497
      0        0        3 1497
      0        0        1 1498
      0        0        2 1498
      0        0        3 1498
      0        0        1 1499
      0        0        2 1499
      0        0        3 1499
      0        0        1 1500
      0        0        2 1500
      0        0        3 1500
      0        0        1 1501
      0        0        2 1501
      0        0        3 1501
      0        0        1 1502
      0        0        2 1502
      0        0        3 1502
      0        0        1 1503
      0        0        2 1503
      0        0        3 1503
      0        0        1 1504
      0        0        2 1504
      0        0        3 1504
      0        0        1 1505
      0        0        2 1505
      0        0        3 1505
      0        0        1 1506
      0        0        2 1506
      0        0        3 1506
      0        0        1 1507
      0        0        2 1507
      0        0        3 1507
      0        0        1 1508
      0        0        2 1508
      0        0        3 1508
      0        0        1 1509
      0        0        2 1509
      0        0        3 1509
      0        0        1 1510
      0        0        2 1510
      0        0        3 1510
      0        0        1 1511
      0        0        2 1511
      0        0        3 1511
      0        0        1 1512
      0        0        2 1512
      0        0        3 1512
      0        0        1 1513
      0        0        2 1513
      0        0        3 1513
      0        0        1 1514
      0        0        2 1514
      0        0        3 1514
      0        0        1 1515
      0        0        2 1515
      0        0        3 1515
      0        0        1 1516
      0        0        2 1516
      0        0        3 1516
      0        0        1 1517
      0        0        2 1517
      0        0        3 1517
      0        0        1 1518
      0        0        2 1518
      0        0        3 1518
      0        0        1 1519
      0        0        2 1519
      0        0        3 1519
      0        0        1 1520
      0        0        2 1520
      0        0        3 1520
      0        0        1 1521
      0        0        2 1521
      0        0        3 1521
      0        0        1 1522
      0        0        2 1522
      0        0        3 1522
      0        0        1 1523
      0        0        2 1523
      0        0        3 1523
      0        0        1 1524
      0        0        2 1524
      0        0        3 1524
      0        0        1 1525
      0        0        2 1525
      0        0        3 1525
      0        0        1 1526
      0        0        2 1526
      0        0        3 1526
      0        0        1 1527
      0        0        2 1527
      0        0        3 1527
      0        0        1 1528
      0        0        2 1528
      0        0        3 1528
      0        0        1 1529
      0        0        2 1529
      0        0        3 1529
      0        0        1 1530
      0        0        2 1530
      0        0        3 1530
      0        0        1 1531
      0        0        2 1531
      0        0        3 1531
      0        0        1 1532
      0        0        2 1532
      0        0        3 1532
      0        0        1 1533
      0        0        2 1533
      0        0        3 1533
      0        0        1 1534
      0        0        2 1534
      0        0        3 1534
      0        0        1 1535
      0        0        2 1535
      0        0        3 1535
      0        0        1 1536
      0        0        2 1536
      0        0        3 1536
      0        0        1 1537
      0        0        2 1537
      0        0        3 1537
      0        0        1 1538
      0        0        2 1538
      0        0        3 1538
      0        0        1 1539
      0        0        2 1539
      0        0        3 1539
      0        0        1 1540
      0        0        2 1540
      0        0        3 1540
      0        0        1 1541
      0        0        2 1541
      0        0        3 1541
      0        0        1 1542
      0        0        2 1542
      0        0        3 1542
      0        0        1 1543
      0        0        2 1543
      0        0        3 1543
      0        0        1 1544
      0        0        2 1544
      0        0        3 1544
      0        0        1 1545
      0        0        2 1545
      0        0        3 1545
      0        0        1 1546
      0        0        2 1546
      0        0        3 1546
      0        0        1 1547
      0        0        2 1547
      0        0        3 1547
      0        0        1 1548
      0        0        2 1548
      0        0        3 1548
      0        0        1 1549
      0        0        2 1549
      0        0        3 1549
      0        0        1 1550
      0        0        2 1550
      0        0        3 1550
      0        0        1 1551
      0        0        2 1551
      0        0        3 1551
      0        0        1 1552
      0        0        2 1552
      0        0        3 1552
      0        0        1 1553
      0        0        2 1553
      0        0        3 1553
      0        0        1 1554
      0        0        2 1554
      0        0        3 1554
      0        0        1 1555
      0        0        2 1555
      0        0        3 1555
      0        0        1 1556
      0        0        2 1556
      0        0        3 1556
      0        0        1 1557
      0        0        2 1557
      0        0        3 1557
      0        0        1 1558
      0        0        2 1558
      0        0        3 1558
      0        0        1 1559
      0        0        2 1559
      0        0        3 1559
      0        0        1 1560
      0        0        2 1560
      0        0        3 1560
      0        0        1 1561
      0        0        2 1561
      0        0        3 1561
      0        0        1 1562
      0        0        2 1562
      0        0        3 1562
      0        0        1 1563
      0        0        2 1563
      0        0        3 1563
      0        0        1 1564
      0        0        2 1564
      0        0        3 1564
      0        0        1 1565
      0        0        2 1565
      0        0        3 1565
      0        0        1 1566
      0        0        2 1566
      0        0        3 1566
      0        0        1 1567
      0        0        2 1567
      0        0        3 1567
      0        0        1 1568
      0        0        2 1568
      0        0        3 1568
      0        0        1 1569
      0        0        2 1569
      0        0        3 1569
      0        0        1 1570
      0        0        2 1570
      0        0        3 1570
      0        0        1 1571
      0        0        2 1571
      0        0        3 1571
      0        0        1 1572
      0        0        2 1572
      0        0        3 1572
      0        0        1 1573
      0        0        2 1573
      0        0        3 1573
      0        0        1 1574
      0        0        2 1574
      0        0        3 1574
      0        0        1 1575
      0        0        2 1575
      0        0        3 1575
      0        0        1 1576
      0        0        2 1576
      0        0        3 1576
      0        0        1 1577
      0        0        2 1577
      0        0        3 1577
      0        0        1 1578
      0        0        2 1578
      0        0        3 1578
      0        0        1 1579
      0        0        2 1579
      0        0        3 1579
      0        0        1 1580
      0        0        2 1580
      0        0        3 1580
      0        0        1 1581
      0        0        2 1581
      0        0        3 1581
      0        0        1 1582
      0        0        2 1582
      0        0        3 1582
      0        0        1 1583
      0        0        2 1583
      0        0        3 1583
      0        0        1 1584
      0        0        2 1584
      0        0        3 1584
      0        0        1 1585
      0        0        2 1585
      0        0        3 1585
      0        0        1 1586
      0        0        2 1586
      0        0        3 1586
      0        0        1 1587
      0        0        2 1587
      0        0        3 1587
      0        0        1 1588
      0        0        2 1588
      0        0        3 1588
      0        0        1 1589
      0        0        2 1589
      0        0        3 1589
      0        0        1 1590
      0        0        2 1590
      0        0        3 1590
      0        0        1 1591
      0        0        2 1591
      0        0        3 1591
      0        0        1 1592
      0        0        2 1592
      0        0        3 1592
      0        0        1 1593
      0        0        2 1593
      0        0        3 1593
      0        0        1 1594
      0        0        2 1594
      0        0        3 1594
      0        0        1 1595
      0        0        2 1595
      0        0        3 1595
      0        0        1 1596
      0        0        2 1596
      0        0        3 1596
      0        0        1 1597
      0        0        2 1597
      0        0        3 1597
      0        0        1 1598
      0        0        2 1598
      0        0        3 1598
      0        0        1 1599
      0        0        2 1599
      0        0        3 1599
      0        0        1 1600
      0        0        2 1600
      0        0        3 1600
      0        0        1 1601
      0        0        2 1601
      0        0        3 1601
      0        0        1 1602
      0        0        2 1602
      0        0        3 1602
      0        0        1 1603
      0        0        2 1603
      0        0        3 1603
      0        0        1 1604
      0        0        2 1604
      0        0        3 1604
      0        0        1 1605
      0        0        2 1605
      0        0        3 1605
      0        0        1 1606
      0        0        2 1606
      0        0        3 1606
      0        0        1 1607
      0        0        2 1607
      0        0        3 1607
      0        0        1 1608
      0        0        2 1608
      0        0        3 1608
      0        0        1 1609
      0        0        2 1609
      0        0        3 1609
      0        0        1 1610
      0        0        2 1610
      0        0        3 1610
      0        0        1 1611
      0        0        2 1611
      0        0        3 1611
      0        0        1 1612
      0        0        2 1612
      0        0        3 1612
      0        0        1 1613
      0        0        2 1613
      0        0        3 1613
      0        0        1 1614
      0        0        2 1614
      0        0        3 1614
      0        0        1 1615
      0        0        2 1615
      0        0        3 1615
      0        0        1 1616
      0        0        2 1616
      0        0        3 1616
      0        0        1 1617
      0        0        2 1617
      0        0        3 1617
      0        0        1 1618
      0        0        2 1618
      0        0        3 1618
      0        0        1 1619
      0        0        2 1619
      0        0        3 1619
      0        0        1 1620
      0        0        2 1620
      0        0        3 1620
      0        0        1 1621
      0        0        2 1621
      0        0        3 1621
      0        0        1 1622
      0        0        2 1622
      0        0        3 1622
      0        0        1 1623
      0        0        2 1623
      0        0        3 1623
      0        0        1 1624
      0        0        2 1624
      0        0        3 1624
      0        0        1 1625
      0        0        2 1625
      0        0        3 1625
      0        0        1 1626
      0        0        2 1626
      0        0        3 1626
      0        0        1 1627
      0        0        2 1627
      0        0        3 1627
      0        0        1 1628
      0        0        2 1628
      0        0        3 1628
      0        0        1 1629
      0        0        2 1629
      0        0        3 1629
      0        0        1 1630
      0        0        2 1630
      0        0        3 1630
      0        0        1 1631
      0        0        2 1631
      0        0        3 1631
      0        0        1 1632
      0        0        2 1632
      0        0        3 1632
      0        0        1 1633
      0        0        2 1633
      0        0        3 1633
      0        0        1 1634
      0        0        2 1634
      0        0        3 1634
      0        0        1 1635
      0        0        2 1635
      0        0        3 1635
      0        0        1 1636
      0        0        2 1636
      0        0        3 1636
      0        0        1 1637
      0        0        2 1637
      0        0        3 1637
      0        0        1 1638
      0        0        2 1638
      0        0        3 1638
      0        0        1 1639
      0        0        2 1639
      0        0        3 1639
      0        0        1 1640
      0        0        2 1640
      0        0        3 1640
      0        0        1 1641
      0        0        2 1641
      0        0        3 1641
      0        0        1 1642
      0        0        2 1642
      0        0        3 1642
      0        0        1 1643
      0        0        2 1643
      0        0        3 1643
      0        0        1 1644
      0        0        2 1644
      0        0        3 1644
      0        0        1 1645
      0        0        2 1645
      0        0        3 1645
      0        0        1 1646
      0        0        2 1646
      0        0        3 1646
      0        0        1 1647
      0        0        2 1647
      0        0        3 1647
      0        0        1 1648
      0        0        2 1648
      0        0        3 1648
      0        0        1 1649
      0        0        2 1649
      0        0        3 1649
      0        0        1 1650
      0        0        2 1650
      0        0        3 1650
      0        0        1 1651
      0        0        2 1651
      0        0        3 1651
      0        0        1 1652
      0        0        2 1652
      0        0        3 1652
      0        0        1 1653
      0        0        2 1653
      0        0        3 1653
      0        0        1 1654
      0        0        2 1654
      0        0        3 1654
      0        0        1 1655
      0        0        2 1655
      0        0        3 1655
      0        0        1 1656
      0        0        2 1656
      0        0        3 1656
      0        0        1 1657
      0        0        2 1657
      0        0        3 1657
      0        0        1 1658
      0        0        2 1658
      0        0        3 1658
      0        0        1 1659
      0        0        2 1659
      0        0        3 1659
      0        0        1 1660
      0        0        2 1660
      0        0        3 1660
      0        0        1 1661
      0        0        2 1661
      0        0        3 1661
      0        0        1 1662
      0        0        2 1662
      0        0        3 1662
      0        0        1 1663
      0        0        2 1663
      0        0        3 1663
      0        0        1 1664
      0        0        2 1664
      0        0        3 1664
      0        0        1 1665
      0        0        2 1665
      0        0        3 1665
      0        0        1 1666
      0        0        2 1666
      0        0        3 1666
      0        0        1 1667
      0        0        2 1667
      0        0        3 1667
      0        0        1 1668
      0        0        2 1668
      0        0        3 1668
      0        0        1 1669
      0        0        2 1669
      0        0        3 1669
      0        0        1 1670
      0        0        2 1670
      0        0        3 1670
      0        0        1 1671
      0        0        2 1671
      0        0        3 1671
      0        0        1 1672
      0        0        2 1672
      0        0        3 1672
      0        0        1 1673
      0        0        2 1673
      0        0        3 1673
      0        0        1 1674
      0        0        2 1674
      0        0        3 1674
      0        0        1 1675
      0        0        2 1675
      0        0        3 1675
      0        0        1 1676
      0        0        2 1676
      0        0        3 1676
      0        0        1 1677
      0        0        2 1677
      0        0        3 1677
      0        0        1 1678
      0        0        2 1678
      0        0        3 1678
      0        0        1 1679
      0        0        2 1679
      0        0        3 1679
      0        0        1 1680
      0        0        2 1680
      0        0        3 1680
      0        0        1 1681
      0        0        2 1681
      0        0        3 1681
      0        0        1 1682
      0        0        2 1682
      0        0        3 1682
      0        0        1 1683
      0        0        2 1683
      0        0        3 1683
      0        0        1 1684
      0        0        2 1684
      0        0        3 1684
      0        0        1 1685
      0        0        2 1685
      0        0        3 1685
      0        0        1 1686
      0        0        2 1686
      0        0        3 1686
      0        0        1 1687
      0        0        2 1687
      0        0        3 1687
      0        0        1 1688
      0        0        2 1688
      0        0        3 1688
      0        0        1 1689
      0        0        2 1689
      0        0        3 1689
      0        0        1 1690
      0        0        2 1690
      0        0        3 1690
      0        0        1 1691
      0        0        2 1691
      0        0        3 1691
      0        0        1 1692
      0        0        2 1692
      0        0        3 1692
      0        0        1 1693
      0        0        2 1693
      0        0        3 1693
      0        0        1 1694
      0        0        2 1694
      0        0        3 1694
      0        0        1 1695
      0        0        2 1695
      0        0        3 1695
      0        0        1 1696
      0        0        2 1696
      0        0        3 1696
      0        0        1 1697
      0        0        2 1697
      0        0        3 1697
      0        0        1 1698
      0        0        2 1698
      0        0        3 1698
      0        0        1 1699
      0        0        2 1699
      0        0        3 1699
      0        0        1 1700
      0        0        2 1700
      0        0        3 1700
      0        0        1 1701
      0        0        2 1701
      0        0        3 1701
      0        0        1 1702
      0        0        2 1702
      0        0        3 1702
      0        0        1 1703
      0        0        2 1703
      0        0        3 1703
      0        0        1 1704
      0        0        2 1704
      0        0        3 1704
      0        0        1 1705
      0        0        2 1705
      0        0        3 1705
      0        0        1 1706
      0        0        2 1706
      0        0        3 1706
      0        0        1 1707
      0        0        2 1707
      0        0        3 1707
      0        0        1 1708
      0        0        2 1708
      0        0        3 1708
      0        0        1 1709
      0        0        2 1709
      0        0        3 1709
      0        0        1 1710
      0        0        2 1710
      0        0        3 1710
      0        0        1 1711
      0        0        2 1711
      0        0        3 1711
      0        0        1 1712
      0        0        2 1712
      0        0        3 1712
      0        0        1 1713
      0        0        2 1713
      0        0        3 1713
      0        0        1 1714
      0        0        2 1714
      0        0        3 1714
      0        0        1 1715
      0        0        2 1715
      0        0        3 1715
      0        0        1 1716
      0        0        2 1716
      0        0        3 1716
      0        0        1 1717
      0        0        2 1717
      0        0        3 1717
      0        0        1 1718
      0        0        2 1718
      0        0        3 1718
      0        0        1 1719
      0        0        2 1719
      0        0        3 1719
      0        0        1 1720
      0        0        2 1720
      0        0        3 1720
      0        0        1 1721
      0        0        2 1721
      0        0        3 1721
      0        0        1 1722
      0        0        2 1722
      0        0        3 1722
      0        0        1 1723
      0        0        2 1723
      0        0        3 1723
      0        0        1 1724
      0        0        2 1724
      0        0        3 1724
      0        0        1 1725
      0        0        2 1725
      0        0        3 1725
      0        0        1 1726
      0        0        2 1726
      0        0        3 1726
      0        0        1 1727
      0        0        2 1727
      0        0        3 1727
      0        0        1 1728
      0        0        2 1728
      0        0        3 1728
      0        0        1 1729
      0        0        2 1729
      0        0        3 1729
      0        0        1 1730
      0        0        2 1730
      0        0        3 1730
      0        0        1 1731
      0        0        2 1731
      0        0        3 1731
      0        0        1 1732
      0        0        2 1732
      0        0        3 1732
      0        0        1 1733
      0        0        2 1733
      0        0        3 1733
      0        0        1 1734
      0        0        2 1734
      0        0        3 1734
      0        0        1 1735
      0        0        2 1735
      0        0        3 1735
      0        0        1 1736
      0        0        2 1736
      0        0        3 1736
      0        0        1 1737
      0        0        2 1737
      0        0        3 1737
      0        0        1 1738
      0        0        2 1738
      0        0        3 1738
      0        0        1 1739
      0        0        2 1739
      0        0        3 1739
      0        0        1 1740
      0        0        2 1740
      0        0        3 1740
      0        0        1 1741
      0        0        2 1741
      0        0        3 1741
      0        0        1 1742
      0        0        2 1742
      0        0        3 1742
      0        0        1 1743
      0        0        2 1743
      0        0        3 1743
      0        0        1 1744
      0        0        2 1744
      0        0        3 1744
      0        0        1 1745
      0        0        2 1745
      0        0        3 1745
      0        0        1 1746
      0        0        2 1746
      0        0        3 1746
      0        0        1 1747
      0        0        2 1747
      0        0        3 1747
      0        0        1 1748
      0        0        2 1748
      0        0        3 1748
      0        0        1 1749
      0        0        2 1749
      0        0        3 1749
      0        0        1 1750
      0        0        2 1750
      0        0        3 1750
      0        0        1 1751
      0        0        2 1751
      0        0        3 1751
      0        0        1 1752
      0        0        2 1752
      0        0        3 1752
      0        0        1 1753
      0        0        2 1753
      0        0        3 1753
      0        0        1 1754
      0        0        2 1754
      0        0        3 1754
      0        0        1 1755
      0        0        2 1755
      0        0        3 1755
      0        0        1 1756
      0        0        2 1756
      0        0        3 1756
      0        0        1 1757
      0        0        2 1757
      0        0        3 1757
      0        0        1 1758
      0        0        2 1758
      0        0        3 1758
      0        0        1 1759
      0        0        2 1759
      0        0        3 1759
      0        0        1 1760
      0        0        2 1760
      0        0        3 1760
      0        0        1 1761
      0        0        2 1761
      0        0        3 1761
      0        0        1 1762
      0        0        2 1762
      0        0        3 1762
      0        0        1 1763
      0        0        2 1763
      0        0        3 1763
      0        0        1 1764
      0        0        2 1764
      0        0        3 1764
      0        0        1 1765
      0        0        2 1765
      0        0        3 1765
      0        0        1 1766
      0        0        2 1766
      0        0        3 1766
      0        0        1 1767
      0        0        2 1767
      0        0        3 1767
      0        0        1 1768
      0        0        2 1768
      0        0        3 1768
      0        0        1 1769
      0        0        2 1769
      0        0        3 1769
      0        0        1 1770
      0        0        2 1770
      0        0        3 1770
      0        0        1 1771
      0        0        2 1771
      0        0        3 1771
      0        0        1 1772
      0        0        2 1772
      0        0        3 1772
      0        0        1 1773
      0        0        2 1773
      0        0        3 1773
      0        0        1 1774
      0        0        2 1774
      0        0        3 1774
      0        0        1 1775
      0        0        2 1775
      0        0        3 1775
      0        0        1 1776
      0        0        2 1776
      0        0        3 1776
      0        0        1 1777
      0        0        2 1777
      0        0        3 1777
      0        0        1 1778
      0        0        2 1778
      0        0        3 1778
      0        0        1 1779
      0        0        2 1779
      0        0        3 1779
      0        0        1 1780
      0        0        2 1780
      0        0        3 1780
      0        0        1 1781
      0        0        2 1781
      0        0        3 1781
      0        0        1 1782
      0        0        2 1782
      0        0        3 1782
      0        0        1 1783
      0        0        2 1783
      0        0        3 1783
      0        0        1 1784
      0        0        2 1784
      0        0        3 1784
      0        0        1 1785
      0        0        2 1785
      0        0        3 1785
      0        0        1 1786
      0        0        2 1786
      0        0        3 1786
      0        0        1 1787
      0        0        2 1787
      0        0        3 1787
      0        0        1 1788
      0        0        2 1788
      0        0        3 1788
      0        0        1 1789
      0        0        2 1789
      0        0        3 1789
      0        0        1 1790
      0        0        2 1790
      0        0        3 1790
      0        0        1 1791
      0        0        2 1791
      0        0        3 1791
      0        0        1 1792
      0        0        2 1792
      0        0        3 1792
      0        0        1 1793
      0        0        2 1793
      0        0        3 1793
      0        0        1 1794
      0        0        2 1794
      0        0        3 1794
      0        0        1 1795
      0        0        2 1795
      0        0        3 1795
      0        0        1 1796
      0        0        2 1796
      0        0        3 1796
      0        0        1 1797
      0        0        2 1797
      0        0        3 1797
      0        0        1 1798
      0        0        2 1798
      0        0        3 1798
      0        0        1 1799
      0        0        2 1799
      0        0        3 1799
      0        0        1 1800
      0        0        2 1800
      0        0        3 1800
      0        0        1 1801
      0        0        2 1801
      0        0        3 1801
      0        0        1 1802
      0        0        2 1802
      0        0        3 1802
      0        0        1 1803
      0        0        2 1803
      0        0        3 1803
      0        0        1 1804
      0        0        2 1804
      0        0        3 1804
      0        0        1 1805
      0        0        2 1805
      0        0        3 1805
      0        0        1 1806
      0        0        2 1806
      0        0        3 1806
      0        0        1 1807
      0        0        2 1807
      0        0        3 1807
      0        0        1 1808
      0        0        2 1808
      0        0        3 1808
      0        0        1 1809
      0        0        2 1809
      0        0        3 1809
      0        0        1 1810
      0        0        2 1810
      0        0        3 1810
      0        0        1 1811
      0        0        2 1811
      0        0        3 1811
      0        0        1 1812
      0        0        2 1812
      0        0        3 1812
      0        0        1 1813
      0        0        2 1813
      0        0        3 1813
      0        0        1 1814
      0        0        2 1814
      0        0        3 1814
      0        0        1 1815
      0        0        2 1815
      0        0        3 1815
      0        0        1 1816
      0        0        2 1816
      0        0        3 1816
      0        0        1 1817
      0        0        2 1817
      0        0        3 1817
      0        0        1 1818
      0        0        2 1818
      0        0        3 1818
      0        0        1 1819
      0        0        2 1819
      0        0        3 1819
      0        0        1 1820
      0        0        2 1820
      0        0        3 1820
      0        0        1 1821
      0        0        2 1821
      0        0        3 1821
      0        0        1 1822
      0        0        2 1822
      0        0        3 1822
      0        0        1 1823
      0        0        2 1823
      0        0        3 1823
      0        0        1 1824
      0        0        2 1824
      0        0        3 1824
      0        0        1 1825
      0        0        2 1825
      0        0        3 1825
      0        0        1 1826
      0        0        2 1826
      0        0        3 1826
      0        0        1 1827
      0        0        2 1827
      0        0        3 1827
      0        0        1 1828
      0        0        2 1828
      0        0        3 1828
      0        0        1 1829
      0        0        2 1829
      0        0        3 1829
      0        0        1 1830
      0        0        2 1830
      0        0        3 1830
      0        0        1 1831
      0        0        2 1831
      0        0        3 1831
      0        0        1 1832
      0        0        2 1832
      0        0        3 1832
      0        0        1 1833
      0        0        2 1833
      0        0        3 1833
      0        0        1 1834
      0        0        2 1834
      0        0        3 1834
      0        0        1 1835
      0        0        2 1835
      0        0        3 1835
      0        0        1 1836
      0        0        2 1836
      0        0        3 1836
      0        0        1 1837
      0        0        2 1837
      0        0        3 1837
      0        0        1 1838
      0        0        2 1838
      0        0        3 1838
      0        0        1 1839
      0        0        2 1839
      0        0        3 1839
      0        0        1 1840
      0        0        2 1840
      0        0        3 1840
      0        0        1 1841
      0        0        2 1841
      0        0        3 1841
      0        0        1 1842
      0        0        2 1842
      0        0        3 1842
      0        0        1 1843
      0        0        2 1843
      0        0        3 1843
      0        0        1 1844
      0        0        2 1844
      0        0        3 1844
      0        0        1 1845
      0        0        2 1845
      0        0        3 1845
      0        0        1 1846
      0        0        2 1846
      0        0        3 1846
      0        0        1 1847
      0        0        2 1847
      0        0        3 1847
      0        0        1 1848
      0        0        2 1848
      0        0        3 1848
      0        0        1 1849
      0        0        2 1849
      0        0        3 1849
      0        0        1 1850
      0        0        2 1850
      0        0        3 1850

22. Attitudes about abortion data set of Table 13.3, in contigency table form


gender poor single any count
1 1 1 1 342
1 1 1 0 26
1 1 0 1 11
1 1 0 0 32
1 0 1 1 6
1 0 1 0 21
1 0 0 1 19
1 0 0 0 356
2 1 1 1 440
2 1 1 0 25
2 1 0 1 14
2 1 0 0 47
2 0 1 1 14
2 0 1 0 18
2 0 0 1 22
2 0 0 0 457

26. Attitudes toward leading crowd data set of Table 13.8


mem1 att1 mem2 att2 count
1 1 1 1 458
1 1 1 0 140
1 1 0 1 110
1 1 0 0 49
1 0 1 1 171
1 0 1 0 182
1 0 0 1 56
1 0 0 0 87
0 1 1 1 184
0 1 1 0 75
0 1 0 1 531
0 1 0 0 281
0 0 1 1 85
0 0 1 0 97
0 0 0 1 338
0 0 0 0 554

27. Data for example in Section 13.4.4 on cluster sampling


 nbhd satis_1 satis_2
1 1 1
1 2 1
1 2 1
1 2 2
1 2 2
2 1 1
2 2 1
2 2 1
2 2 2
2 2 2
3 1 2
3 1 2
3 2 2
3 2 2
3 3 2
4 1 2
4 2 1
4 2 1
4 2 2
4 3 1
5 2 2
5 2 2
5 2 2
5 2 2
5 3 2
6 1 1
6 2 1
6 2 1
6 2 1
6 2 2
7 1 1
7 1 1
7 1 1
7 2 2
7 3 2
8 1 1
8 2 1
8 2 2
8 2 2
8 2 2
9 1 1
9 1 1
9 1 1
9 3 1
9 3 3
10 1 2
10 2 2
10 2 2
10 2 2
10 2 3
11 1 1
11 1 2
11 2 2
11 2 2
11 3 1
12 1 2
12 2 1
12 2 1
12 2 1
12 2 1
13 2 1
13 2 1
13 2 1
13 2 1
13 2 2
14 2 1
14 2 2
14 2 2
14 3 3
14 3 3
15 1 1
15 1 1
15 2 1
15 2 1
15 2 2
16 1 1
16 2 1
16 2 2
17 2 1
17 2 2
17 2 3
17 3 2
17 3 2
18 2 1
18 2 3
18 3 3
19 1 1
19 1 1
19 2 1
19 2 1
19 2 2
20 1 1
20 1 1
20 2 1
20 2 1
20 3 1

28. Clinical trials data set for Exercise 13.17


 Center  Treatment   Much_Better  Better  Unchanged/Worse 
  1  Drug      13  7  6
 1   Placebo    1  1   10 
 2  Drug       2  5   10 
 2   Placebo    2  2   1  
 3  Drug      11  23  7  
 3   Placebo    2  8   2  
 4  Drug       7  11  8  
 4   Placebo    0  3   2  
 5  Drug      15  3   5  
 5   Placebo    1  1   5  
 6  Drug      13  5   5  
 6   Placebo    4  0   1  
 7  Drug       7  4   13 
 7   Placebo    1  1   11 
 8  Drug      15  9   2  
 8   Placebo    3  2   2  

29. Data for Exercise 14.15 on Buchanan vote in 2000


  county      perot      total     buchanan    total
              vote     vote 1996     vote     vote 2000

  Alachua      8072      74484       262        84839
  Baker         667       6634        73         8128
  Bay          5922      51566       248        58563
  Bradford      819       8247        65         8638
  Brevard     25249     195055       570       217543
  Broward     38964     505015       789       571685
  Calhoun       630       4158        90         5157
  Charlott     7783      63014       182        66715
  Citrus       7244      49585       270        56940
  Clay         3281      47040       186        57116
  Collier      6320      72511       122        91873
  Columbia     1970      16326        89        18358
  Desoto        965       7485        36         7771
  Dixie         652       3795        29         4627
  Duval       13844     253943       652       263371
  Escambia     8587     107687       504       116220
  Flagler      2185      20075        83        27017
  Franklin      878       4569        33         4618
  Gadsden       938      14193        39        14493
  Gilchris      841       4795        29         5336
  Glades        521       3431         9         3346
  Gulf         1054       5986        71         6104
  Hamilton      406       3670        23         3928
  Hardee        851       6204        30         6210
  Hendry       1135       8896        22         8112
  Hernando     7272      58055       242        65033
  Highland     3739      33699       127        35045
  Hillsbor    25154     308190       847       351913
  Holmes       1208       6801        76         7306
  IndianRi     4635      43963       105        49458
  Jackson      1602      15509       102        16246
  Jefferso      393       4808        29         5624
  Lafayett      316       2322        10         2493
  Lake         8813      73911       289        88266
  Lee         18389     165923       305       183593
  Leon         6672      91685       282       102692
  Levy         1774      11065        67        12614
  Liberty       376       1342        39         2385
  Madison       578       5584        29         6132
  Manatee     10360      96741       272       109878
  Marion      11340      90146       563       102178
  Martin       5005      54646       108        61666
  MiamiDad    24722     553491       560       624168
  Monroe       4817      32450        47        33679
  Nassau       1657      21159        90        23502
  Okaloosa     5432      62963       267        70293
  Okeechob     1666       9936        43         9819
  Orange      18191     231061       446       278918
  Osceola      6091      46484       145        55270
  PalmBeac    30739     397231      3407       430762
  Pasco       18011     133457       570       142108
  Pinellas    36990     376218      1010       396092
  Polk        14991     150140       538       167676
  Putnam       3272      25145       148        26074
  SantaRos     4957      42336       311        50111
  Sarasota    14939     148950       305       160327
  Seminole     9357     114878       194       136315
  StJohns      4205      48539       229        60494
  StLucie      8482      73897       124        77756
  Sumter       2375      15397       114        22184
  Suwannee     1874      12144       108        12369
  Taylor       1140       7997        27         6791
  Union         425       3462        29         3800
  Volusia     17319     160118       396       182109
  Wakulla      1091       7165        46         8545
  Walton       2342     15514        120       18209 
  Washingt     1287      7859         88        7960 

30. Election data set of Table 15.5 (1 = Dem, 0 = Rep)


State  e1  e2  e3  e4  e5  e6  e7  e8 
  Alab 0 0 0 0 0 0 0 0  
  Alas 0 0 0 0 0 0 0 0  
  Ariz 0 0 0 0 1 0 0 0  
  Arka 0 0 0 1 1 0 0 0  
  Cali 0 0 0 1 1 1 1 1  
  Colo 0 0 0 1 0 0 0 1  
  Conn 0 0 0 1 1 1 1 1  
  Dela 0 0 0 1 1 1 1 1  
  DisC 1 1 1 1 1 1 1 1  
  Flor 0 0 0 0 1 0 0 1  
  Geor 1 0 0 1 0 0 0 0  
  Hawa 1 0 1 1 1 1 1 1  
  Idah 0 0 0 0 0 0 0 0  
  Illi 0 0 0 1 1 1 1 1  
  Indi 0 0 0 0 0 0 0 1  
  Iowa 0 0 1 1 1 1 0 1  
  Kans 0 0 0 0 0 0 0 0  
  Kent 0 0 0 1 1 0 0 0  
  Loui 0 0 0 1 1 0 0 0  
  Main 0 0 0 1 1 1 1 1  
  Mary 1 0 0 1 1 1 1 1  
  Mass 0 0 1 1 1 1 1 1  
  Mich 0 0 0 1 1 1 1 1  
  Minn 1 1 1 1 1 1 1 1  
  Miss 0 0 0 0 0 0 0 0  
  Miso 0 0 0 1 1 0 0 0  
  Mont 0 0 0 1 0 0 0 0  
  Nebr 0 0 0 0 0 0 0 0  
  Neva 0 0 0 1 1 0 0 1  
  NewH 0 0 0 1 1 0 1 1  
  NewJ 0 0 0 1 1 1 1 1  
  NewM 0 0 0 1 1 1 0 1  
  NewY 0 0 1 1 1 1 1 1  
  NorC 0 0 0 0 0 0 0 1  
  NorD 0 0 0 0 0 0 0 0  
  Ohio 0 0 0 1 1 0 0 1  
  Okla 0 0 0 0 0 0 0 0  
  Oreg 0 0 1 1 1 1 1 1  
  Penn 0 0 0 1 1 1 1 1  
  Rhod 1 0 1 1 1 1 1 1  
  SouC 0 0 0 0 0 0 0 0  
  SouD 0 0 0 0 0 0 0 0  
  Tenn 0 0 0 1 1 0 0 1  
  Texa 0 0 0 0 0 0 0 0  
  Utah 0 0 0 0 0 0 0 0  
  Verm 0 0 0 1 1 1 1 1  
  Virg 0 0 0 0 0 0 0 1  
  Wash 0 0 1 1 1 1 1 1  
  WesV 1 0 1 1 1 0 0 0  
  Wisc 0 0 1 1 1 1 1 1  
  Wyom 0 0 0 0 0 0 0 0  

31. Grounds for divorce data set of Table 15.6 for Exercise 15.10


state incompat cruelty desertn non_supp alcohol felony impotenc insanity separate 
 Alabama  1 1 1 1 1 1 1 1 1 
 Alaska  1 1 1 0 1 1 1 1 0   
 Arizona  1 0 0 0 0 0 0 0 0 
 Arkansas  0 1 1 1 1 1 1 1 1 
 California  1 0 0 0 0 0 0 1 0 
 Colorado  1 0 0 0 0 0 0 0 0 
 Connecticut  1 1 1 1 1 1 0 1 1 
 Delaware  1 0 0 0 0 0 0 0 1 
 Florida  1 0 0 0 0 0 0 1 0 
 Georgia  1 1 1 0 1 1 1 1 0 
 Hawaii  1 0 0 0 0 0 0 0 1 
 Idaho  1 1 1 1 1 1 0 1 1 
 Illinois  0 1 1 0 1 1 1 0 0 
 Indiana  1 0 0 0 0 1 1 1 0 
 Iowa  1 0 0 0 0 0 0 0 0 
 Kansas  1 1 1 0 1 1 1 1 0 
 Kentucky  1 0 0 0 0 0 0 0 0 
 Louisiana  0 0 0 0 0 1 0 0 1 
 Maine  1 1 1 1 1 0 1 1 0  
 Maryland  0 1 1 0 0 1 1 1 1 
 Massachusetts  1 1 1 1 1 1 1 0 1 
 Michigan  1 0 0 0 0 0 0 0 0 
 Minnesota  1 0 0 0 0 0 0 0 0 
 Mississippi  1 1 1 0 1 1 1 1 0 
 Missouri  1 0 0 0 0 0 0 0 0 
 Montana  1 0 0 0 0 0 0 0 0 
 Nebraska  1 0 0 0 0 0 0 0 0 
 Nevada  1 0 0 0 0 0 0 1 1 
 NewHampshire  1 1 1 1 1 1 1 0 0 
 NewJersey  0 1 1 0 1 1 0 1 1 
 NewMexico  1 1 1 0 0 0 0 0 0 
 NewYork  0 1 1 0 0 1 0 0 1 
 NorthCarolina  0 0 0 0 0 0 1 1 1 
 NorthDakota  1 1 1 1 1 1 1 1 0 
 Ohio  1 1 1 0 1 1 1 0 1 
 Oklahoma  1 1 1 1 1 1 1 1 0 
 Oregon  1 0 0 0 0 0 0 0 0 
 Pennsylvania  0 1 1 0 0 1 1 1 0 
 RhodeIsland  1 1 1 1 1 1 1 0 1 
 SouthCarolina  0 1 1 0 1 0 0 0 1 
 SouthDakota  0 1 1 1 1 1 0 0 0 
 Tennessee  1 1 1 1 1 1 1 0 0 
 Texas  1 1 1 0 0 1 0 1 1 
 Utah  0 1 1 1 1 1 1 1 0 
 Vermont  0 1 1 1 0 1 0 1 1 
 Virginia  0 1 0 0 0 1 0 0 1 
 Washington  1 0 0 0 0 0 0 0 1 
 WestVirginia  1 1 1 0 1 1 0 1 1 
 Wisconsin  1 0 0 0 0 0 0 0 1 
 Wyoming  1 0 0 0 0 0 0 1 1 

Copyright © 2012, Alan Agresti, Department of Statistics, University of Florida.


 


  • {\infopagename}

next up previous
Next:

Alan Agresti 2001-12-27

Thompson, LA – S-Plus and R manual to accompany Agresti (2002) – Ch 02

Before we get into logistic regression and related approaches to modeling categorical data, let’s examine some of the fundamental of modeling contingency tables.


Agresti (2002) Ch 01 covers distributions and inference for categorical data – we’re going to skip over this material.


Agresti (2002) Ch 02 – which introduces I ×J (two-way) contingency tables:

Row variable and column variable – marginal distribution and conditional distribution

If both the row and column of a table denote random variables, then the probabilities {πij} define the joint distribution of the two variables.  The marginal distributions are denoted by {πi+} for the row variable and {π+ j} for the column variable.  For a fixed value i of the row variable, the column variable has the conditional distribution1|i ,…,πJ i| }.  The conditional distribution is especially important if the row variable is fixed by design (i.e.  not free to vary for each observation).

Independence of row variable and column variable

Row and column variables are independent if the conditional distribution of the column variable given the row variable is the same as the marginal distribution of the column variable (and vice versa).  That is, πj i| + j for i = 1,…, I, and πi j| i+ j = 1,…, J.  Equivalently, if all joint probabilities equal the product of their marginal probabilities: πij =ππi+ + j , for all i and j.  Thus, when the two variables are independent, knowledge of the value of the row variable does not change the distribution of the column variable, and vice versa.

Independence of explanatory variable and response variable

When the row variable is an explanatory variable and the column is a response variable, then there is no joint distribution, and independence is referred to as homogeneity of the conditional distributions of the column variable given a value for the row variable.


Maybe a little esoteric:

Sampling scheme should determine the distribution of cell counts

The distributions of the cell counts {Yij} differ depending on how sampling was done.

  • If observations are to be collected over a certain period of time and cross-classified into one of the I ×J categories, then a Poisson sampling model might be used where cell counts are treated as independent Poisson random variables with parameters {µij }.
  • If the total sample size of observations is fixed in advance (e.g., in a cross-sectional study), then a multinomial sampling model might be used where cell counts are treated as multinomial random variables with index n and probabilities {πij}.
  • If the row totals are fixed in advance, perhaps as fixed-size random samples drawn from specific populations that are to be compared, as in prospective studies, then a product-multinomial sampling model may apply where for each i, the counts {Yj i| } have a multinomial distribution with index ni and probabilities πj i| j = 1,…, J .
  • If both row and column totals are fixed by design, then a hypergeometric sampling distribution applies for the cell counts.

Sampling scheme often does not determine the distribution of cell counts
However, there are times when certain sampling models are assumed, but sampling was actually done differently.  For example, when the row variable is an explanatory variable, product multinomial sampling model may be used even though the row totals were not actually fixed in advance.  Also, the Poisson model is used even when the total sample size is fixed in advance.


Section 2.2 discusses comparing two proportions from two samples, including the difference of proportions, relative risk, and odds ratio.

Without offering an explanation, odds ratio is the best comparison to use.

Odds ratio

The odds ratio is the ratio of odds of a positive response by group

θ= [π1|1/(1−π1|1)] / [π1|2/(1−π1|2)] = [π11π22]/[π1|2π2|1 ]

When θ= 1, the row and column variables are independent.  Values of θ farther from 1.0 in a given direction represent stronger association.  The odds ratio can be used with a joint distribution of the row and column variables too.  Indeed, it can be used with prospective (rows totals fixed), retrospective (column totals fixed), and cross-sectional designs.  Finally, if the rows and columns are interchanged, the value of the odds ratio does not change.  The sample odds ratio uses the observed sample counts, nij.

Confounding explanatory variables and conditional association

In observational studies, confounding variables can be controlled with stratification or conditioning.  The association between two variables X and Y given that another measured variable Z takes the value z is called a conditional association.  The 2 x 2 table resulting from cross-classifying all observations with Z = z by their X and Y values is called a partial table.  If Z is ignored, the X-Y table is called a marginal table.


Simpson’s Paradox

Simpson’s Paradox is the result that a marginal association can have a different direction than a conditional association.  For example, in the death penalty example on p. 49-51, ignoring victim’s race, the death penalty (Y) is more likely for whites than for blacks (X).  However, conditioning on victim’s race (either black or white), the death penalty is more likely for blacks than for whites.  The paradox in this case can be explained by the strong association between victim’s race (ignored in the marginal association) and defendant’s race and that between victim’s race and the death penalty.  The death penalty was more likely when the victims were white (regardless of defendant race).  Also, whites were more likely to kill whites than any other victim/defendant race combination in the sample.  So, there are a lot of whites receiving the death penalty in the sample.  On the other hand, blacks were more likely to kill blacks.  Thus, there are fewer blacks receiving the death penalty in the sample.  But, if we look at only white victims, there are relatively more blacks receiving the death penalty than whites.  The same is true for black victims.  An unmodeled association between victim’s and defendant’s race hides this conclusion.

Does Simpson’s Paradox imply that we should distrust all contingency table analyses?  After all, there are undoubtedly unmeasured variables that could be potential conditioning variables in all contingency tables.  Could these variables change the direction of marginal associations?  Page 51 in Agresti paraphrases J. Cornfield’s result “that with a very strong XY association [marginal association], a very strong association must exist between the confounding variable Z and both X and Y in order for the effect to disappear or change …”.


Conditional independence

For I x J x K tables (where X has I levels, Y has J levels, and Z has K levels), if X and Y are independent in partial table k, then X and Y are conditionally independent given that Z takes on value k. If X and Y are independent at all levels of Z, then X and Y are conditionally independent given Z.

Conditional independence does not imply marginal independence.  For 2 x 2 x K tables, X and Y are conditionally independent given Z if the odds ratio between X and Y equals 1 at each category of Z.  For the general case of I x J x K tables, independence is equivalent to all (I −1)(J −1) local odds ratios equaling 1.0.


An analogy to no three-way interaction in ANOVA is homogeneous association.  A 2 x 2 x K table has homogeneous XY association if the conditional odds ratios comparing two categories of X to two categories of Y are the same at each level of Z.  When interaction exists, the conditional odds ratio for any pair of variables (say X and Y) changes across categories of the third (say Z), wherein the third variable is called an effect modifier because the effect of X on Y (the response) changes depending on the level of Z.  For the general case of I x J x K tables, homogeneous XY association means that any conditional odds ratio formed using two categories of X and two categories of Y is the same at each category of Z.


Summary measures of association – nominal data

  • Kendall and Stuart’s measure of proportional reduction in variance from the marginal distribution of the response to the conditional distributions given the value of an explanatory vector; and
  • Theil’s uncertainty coefficient – the proportional reduction in entropy (or uncertainty) in response given explanatory variables.

Summary measures of association – ordinal data

  • concordance, and
  • Gamma.

Bohanec, Marko – (DEXi 5.1) – 2015

01a – DEXi: A Program for Multi-Attribute Decision Making Version 5.01

Purpose

DEXi is a computer program for multi-attribute decision making. It is aimed at interactive development of qualitative multi-attribute decision models and the evaluation of options. This is useful for supporting complex decision-making tasks, where there is a need to select a particular option from a set of possible ones so as to satisfy the goals of the decision maker. A multi-attribute model is a hierarchical structure that represents the decomposition of the decision problem into subproblems, which are smaller, less complex and possibly easier to solve than the complete problem.

Further information on DEXi:

Functionality
Screenshots
Documentation
Development and history
Typical applications

Download

DEXi is implemented in Delphi and runs on Microsoft Windows platforms. It can be used free of charge.

The latest DEXi version is 5.01 and is available in two languages:

Slovene: DEXi501si_setup.exe
English: DEXi501en_setup.exe

Related software

  • DEX is the predecessor of DEXi.
  • JDEXi is an open-source Java library implementing: parsing of DEXi models and evaluation of options.
  • DEXiTree: a program for pretty drawing of DEXi trees.
  • DEXiEval: a command-line utility program for batch evaluation of options using a DEXi model.

01b – DEXi Functionality

DEXi supports two basic tasks:

  1. the development of qualitative multi-attribute models;
  2. the application of models for the evaluation and analysis of options.

The models are developed by defining:

  • attributes: qualitative variables that represent decision subproblems,
  • scales: ordered or unordered sets of symbolic values that can be assigned to attributes,
  • tree of attributes: a hierarchical structure representing the decomposition of the decision problem,
  • utility functions: rules that define the aggregation of attributes from bottom to the top of the tree of attributes.

In the evaluation and analysis stage, DEXi facilitates:

  • description of options: defining the values of basic attributes (terminal nodes of the tree),
  • evaluation of options: a bottom up aggregation of option values based on utility functions,
  • analysis of options: what-if analysis, “plus-minus-1” analysis, selective explanation and comparison of options,
  • reporting: graphical and textual presentation of models, options and evaluation results.

DEXi differs from most conventional multi-attribute decision modeling tools in that it uses qualitative (symbolic) attributes instead of quantitative (numeric) ones. Also, aggregation (utility) functions in DEXi are defined by if-then decision rules rather numerically by weights or some other kind of formula. (However, DEXi does support weights indirectly.)

In comparison with its predecessor DEX, DEXi has a more modern and more convenient user interface. Also, it has better graphical and reporting capabilities, and facilitates the use of weights to represent and assess qualitative utility functions. On the other hand, DEXi is somewhat less powerful than DEX in dealing with incomplete option descriptions: DEX employs probabilistic and fuzzy distribution of values, while DEXi facilitates only the use of crisp or unknown option values.

01c – DEXi Screenshots

Editing a decision model

Model Editing

Editing a qualitative attribute scale

Scale Editing

Defining decision rules

Rules Editing

Editing option descriptions

Options Editing

Option evaluation and analysis

Option Evaluation

Displaying charts

Chatrs

Report preview

Report Preview

01d – DEXi Documentation and Publications

Both DEXi installation packages, Slovene and English, include an English help file.

Documentation in Slovene

An early DEXi User’s Manual is available as:

Jereb, E., Bohanec, M., Rajkovič, V.: DEXi: Računalniški program za večparametrsko odločanje, Moderna organizacija, Kranj, 2003.

Further information on decision analysis, multi-attribute modeling, fundamental DEXi concepts and underlying methods is available in:

Bohanec, M.: Odločanje in modeli. DMFA – založništvo, 1. ponatis, 2012. [O knjigi…]

Documentation in English

The English help file, which is distributed with the installation, is up-to-date and describes DEXi version 4.01.

The DEXi 5.00 User’s Manual in English is available as:

Bohanec, M.: DEXi: Program for Multi-Attribute Decision Making, User’s Manual, Version 5.00. IJS Report DP-11897, Jožef Stefan Institute, Ljubljana, 2015.
[Also available: a printer-friendly version without hyperlinks.]

Selected Publications

M. Bohanec, V. Rajkovič: Večparametrski odločitveni modeli. Organizacija 28(7), 427-438, 1995.

Bohanec, M., Rajkovič, V.: Multi-attribute decision modeling: Industrial applications of DEX. Informatica 23, 487-491, 1999.

Bohanec, M., Zupan, B., Rajkovič, V.: Applications of qualitative multi-attribute decision models in health care, International Journal of Medical Informatics 58-59, 191-205, 2000.

Cestnik, B., Bohanec, M.: Decision support in housing loan allocation: A case study, IDDM-2001: ECML/PKDD-2001 Workshop Integrating Aspects of Data Mining, Decision Support and Meta-Learning: Positions, Developments and Future Directions (eds. Giraud-Carrier, C., Lavrač, N., Moyle, S., Kavšek, B.), Freiburg, 21-30, 2001.

Mladenić, D., Lavrač, N., Bohanec, M., Moyle, S. (eds.): Data mining and decision support: Integration and collaboration. Kluwer Academic Publishers, 2003. Chapters:

  • Bohanec, M.: Decision support. 23-35.
  • Bohanec, M., Rajkovič, V., Cestnik, B.: Five decision support applications. 177-189.
  • Moyle, S., Bohanec, M., Ostrowski, E.: Large and tall buildings: A case study in the application of decision support and data mining. 191-202.

Moyle, S., Ostrowski, E., Bohanec, M.: Knowledge development using data mining: A specific application in the construction industry. Leveraging corporate knowledge (ed. Truch, E.), Gower, 181-197, 2004.

Vintar, M., Grad, J. (ur.): E-uprava: Izbrane razvojne perspektive, Univerza v Ljubljani, Fakulteta za upravo, 2004.:

  • Leben, A., Bohanec, M.: Vrednotenje portalov življenjskih situacij, 123-140.
  • Bohanec, M.: Odločanje in večparametrsko modeliranje, 205-219.

Bohanec, M., Džeroski, S., Žnidaršič, M., Messéan, A., Scatasta, S., Wesseler, J.: Multi-attribute modeling of economic and ecological impacts of cropping systems, Informatica 28, 387-392, 2004.

Leben, A., Kunstelj, M., Bohanec, M., Vintar, M.: Evaluating public administration e-portals. Information Polity 21(3/4), 207-225, 2006.

Bohanec, M., Messéan, A., Angevin, F., Žnidaršič, M.: SMAC Advisor: A decision-support tool on coexistence of genetically-modified and conventional maize. Proc. Information Society IS 2006, Ljubljana, 9-12, 2006

Verdev. M., Bohanec, M., Džeroski, S.: Decision support for a waste electrical and electronic equipment treatment system. Proc. Information Society IS 2006, Ljubljana, 89-92, 2006

Taškova, K., Stojanova, D., Bohanec, M., Dleroski, S.: A qualitative decision-support model for evaluating researchers. Informatica 31(4), 479-486, 2007.

Omerčević, D., Zupančič, M., Bohanec, M., Kastelic, T.: Intelligent response to highway traffic situations and road incidents. Proc. TRA 2008, Transport Research Arena Europe 2008, 21-24 April 2008, Ljubljana, Slovenia (ed. A. Žnidarič). Ljubljana: DDC svetovanje inženiring: ZAG, Zavod za gradbeništvo Slovenije: DRC, Družba v cestni in prometni stroki Slovenije, 1-6, 2008.

Žnidaršič, M., Bohanec, M., Kok, E.J., Prins, T.W.: Qualitative risk assessment for adventitious presence of unauthorized genetically modified organisms. Proceedings of ISIT 2009, 1st International Conference on Information Society and Information Technologies, Novo mesto: Faculty of information studies. 12.-13.10.2009, Dolenjske Toplice, 7 p., 2009.

Žnidaršič, M., Bohanec, M., Lavrač, N., Cestnik, B.: Project self-evaluation methodology: The Healthreats project case study. Proc. Information Society IS 2009, Ljubljana, 85-88, 2009.

Bohanec, M., Žnidaršič, M.: Izkušnje z večparametrskimi odločitvenimi modeli pri podpori odločanja o gensko spremenjenih organizmih. DAES 2010: Sodobni izzivi menedžmenta v agroživilstvu (ur. Č. Rozman, S. Kavčič), Pivola, 18.-19.3.2010, 29-37, 2010.

Marinič, S., Bohanec, M.: Večparametrsko vrednotenje variant v odvisnosti od konteksta: Model za vrednotenje strešnih kritin Proceedings of the 15th International Conference Information Society IS 2012, 8.-12.10.2012, Ljubljana, 76-79, 2012.

Bohanec, M., Rajkovič, V., Bratko, I., Zupan, B., Žnidaršič, M.: DEX methodology: Three decades of qualitative multi-attribute modelling. Informatica 37, 49-54, 2013.

Alić, I., Siering, M., Bohanec, M.: Hot stock or not? A qualitative multi-attribute model to detect financial market manipulation. eInnovation: Challenges and impacts for individuals, organizations and society, Proceedings of 26th Bled eConference (ed. D.L.Wigand), June 9-13, 2013, Bled, Slovenia, Kranj: Moderna organizacija, 64-77, 2013.

Trdin, N., Bohanec, M., Janža, M.: Decision support system for management of water sources. Proceedings of the 16th International Conference Information Society IS 2013, 7.-11.10.2013, Ljubljana, 118-121, 2013.

Bohanec, M., Aprile, G., Costante, M., Foti, M., Trdin, N.: A hierarchical multi-attribute model for bank reputational risk assessment. DSS 2.0 — Supporting Decision Making with New Technologies (eds. Phillips-Wren, G., Carlsson, S., Respício, A., Brézillon, P.), Amsterdam: IOS Press, ISBN 978-1-61499-398-8, 92-103, 2014.

Mileva Boshkoska, B., Bohanec, M., Boškoski, P., Juričić, Đ.: Copula-based decision support system for quality ranking in the manufacturing of electronically commutated motors. Journal of Intelligent Manufacturing 26, 281-293, 2015.

Bohanec, M., Delibašić, B.: Data-mining and expert models for predicting injury risk in ski resorts. Decision Support Systems V – Big Data Analytics for Decision Making, First International Conference ICDSST 2015, Belgrade, Serbia, May 27-29, 2015, Springer, 46-60, 2015.

01d – DEXi Applications

DEXi is particularly suitable for solving complex decision problems, which typically involve:

  • many (say, 15 or more) attributes,
  • many options (10 or more),
  • judgment, which prevalently requires qualitative reasoning rather than numerical evaluation,
  • inaccurate and/or missing data,
  • group decision making, which requires communication and explanation.

For successful application, DEXi requires sufficient resources, particularly expertise and time for developing a DEXi model.

Some typical application areas and decision problems, in which DEX and DEXi have been used so far, are the following:

  1. Information technology
    • evaluation of computers
    • evaluation of software
    • evaluation of Web portals
  2. Projects
    • evaluation of projects
    • evaluation of proposals and investments
    • product portfolio evaluation
  3. Companies
    • business partner selection
    • performance evaluation of companies
  4. Personnel Management
    • personnel evaluation
    • selection and composition of expert groups
    • evaluation of personal applications for jobs
  5. Medicine and Health-Care
    • risk assessment
    • diagnosis and prognosis
  6. Other Areas
    • assessment of technologies
    • assessments in ecology and environment
    • granting personal/corporate loans

02 – JDEXi: Open-source DEXi Java Library Version 3.0

Purpose

JDEXi3.zip contains a library of open-source Java classes that implement the evaluation of decision alternatives based on qualitative multi-attribute models produced by DEXi software.

JDEXi (version 3) supports:

  • parsing and reading DEXi models from .dxi files or strings (XML format) [constructor Model()]
  • obtaining information about model attributes and attribute scales [methods getAttribute*(), getScale*(), …]
  • obtaining information about utility functions and decision rules [methods getRule*(), rule*(), function*(), getFunctionString(), …]
  • clearing and setting model input values [methods setInputValue(s), …]
  • carrying out the evaluation [methods evaluate(), …]
  • obtaining evaluation results [methods getOutputValue(s), …]
  • modification of decision rules [method getFunctionString()]

JDEXi3 supports only a fairly limited modification of decision rules. DEXi software should be used for any more extensive modification of models.


Authors: Marko Bohanec, Dušan Omerčević, Andrej Kogovšek

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

Contents of JDEXi3.zip

bin/ – contains compiled java files
src/ – contains java source files
doc/ – contains javadoc
JDEXi3.jar – JDEXi classes; see javadoc for the list of all classes
JDEXi3Eval.jar – runnable jar for running JDEXi (command-line)
TestJDEXi3.jar and DumpFile.jar – runnable jars for showing the use of JDEXi library (see the sources in src/test/)
Licence.txt
lesser.txt – GNU LESSER GENERAL PUBLIC LICENSE
Car.dxi – a sample DEXi file (car selection demo)
readme.html – this file

JDEXi3Eval.jar

Runnable jar for running JDEXi application through the command line. It takes 2 mandatory parameters:

  1. DEXi_file_name: file containing a DEXi model (.dxi)
  2. Variables: a “;”-separated list of name=value pairs

Example

java -jar JDEXi3Eval.jar Car.dxi "BUY.PRICE=low;MAINT.PRICE=low;#PERS=more;#DOORS=more;LUGGAGE=big;SAFETY=high"

TestJDEXi3.jar

Loads a DEXi model, displays some of its elements (attributes, value scales, decision rules) and evaluates a random decision alternative.

java -jar TestJDEXi3.jar Car.dxi

DumpFunctions.jar

Loads a DEXi model and prints out all utility functions it contains.

java -jar DumpFunctions.jar Car.dxi

03 – DEXiTree: A Program for Pretty Drawing of DEXi Trees Version 0.94

Purpose

DEXiTree is a companion program to DEXi, aimed at making nice drawings of DEXi’s trees of attributes. Actually, DEXiTree is quite a general and powerful tree-drawing program that:

  • offers four different tree-drawing algorithms (called “Distribute”, “Align”, “Walker”, and “QP”);
  • draws trees in four different directions (Top-Down, Left-Right, Bottom-Up and Right-Left);
  • provides an extensive set of parameters for controlling the appearance of trees and their components.

Please see some DEXiTree screenshots.

Download

DEXiTree is implemented in Delphi and is available for Microsoft Windows. The latest version is 0.94 and is compatible with DEXi 4.01 and later.

Download DEXiTree Version 0.94: DEXiTree094.zip

No installation is required; just unpack the zip file and run DEXiTree.exe.

Usage

DEXiTree is typically used in the following steps:

  1. load (File/Open) a DEXi model from a .dxi file;
  2. interactively alter DEXiTree’s drawing parameters until you are satisfied with the drawing;
  3. save the drawing to a file (File/Save as…) or transfer it to other applications through the clipboard (Edit/Copy).

Drawings can be saved and/or transferred in two different graphic formats:

  • Windows Enhanced Metafile (.emf) [vector graphic format], or
  • Windows Bitmap (.bmp) [bitmap graphic format].

DEXiTree uses its own XML-based “DEXiTree file” format (.dxt) for representing the currently drawn tree of attributes and corresponding drawing parameters, DEXiTree can both load (File/Open) and save (File/Save as…) such files. Loading can be selective so that only a tree structure or only drawing parameters are loaded from file, leaving the other component intact.

See the file DEXiTree.txt for more detailed instructions and conditions of use.

If you use this software for any purpose, an acknowledgment/citation in your product and an informative message to the author would be appreciated.

Copyright © 2007-2015 Marko Bohanec. All rights reserved.

Notes

Editing of trees is not supported in DEXiTree. Use DEXi to modify the structure of trees.

DEXiTree has been provided in the hope that it will be useful, but without warranty of any kind.

Any feedback on DEXiTree will be greatly appreciated. Please send your comments, suggestions, bug reports, etc., to Marko Bohanec.

Publications

04 – DEXiEval: Command-Line Utility for Batch Evaluation of DEXi Options

Purpose

DEXiEval is a command-line utility program for batch evaluation of options (decision alternatives) using a DEXi model. Basically, DEXiEval reads a DEXi model from a DEXi file and loads option data from another input file. It evaluates these options and writes the evaluation results to output option data files. In one turn, DEXiEval can create several output files in different formats.

Download

DEXiEval is implemented in Delphi and is available for Microsoft Windows and Linux. The latest version is 4.0 and is compatible with DEXi 2.0, DEXi 3.0, DEXi 4.0 and possibly later.

Windows: DEXiEval40.zip
Linux: DEXiEval40.tgz

No installation is required; just unpack the archive and run DEXiEval.

Usage

See the file DEXiEval.txt for detailed instructions and conditions of use.

If you use this software for any purpose, an acknowledgment in your product and an informative message to the author would be appreciated.

Ramik, Jaroslav – Decision Analysis Module for Excel (DAME) – 2014

01 – DAME Tool – Group Decision Making and Evaluation Made Simple

Group evaluation techniques are many and group decision-making is always challenging. There are whole software packages today specializing only in just that like Expert Choice, for example. But what if you don’t want to shell out thousands of dollars and achieve a comparable result? Believe it or not, there is a way and it is rather simple and virtually free. DAME stands for Decision Analysis Module for Excel and it is quite a useful solution.

Group evaluation techniques are many and group decision-making is always challanging. There are whole software packages today specializing only in just that like Expert Choice, for example. But what if you don’t want to shell out thousands of dollars and achieve a comparable result? Believe it or not, there is a way and it is rather simple and virtually free. DAME stands for Decision Analysis Module for Excel and it is quite cool.

Take an example of Expert Choice. Within a space of two decades it grew from a niche decision making software application to a big group decision making tool. It is based on AHP (Analytical Hierarchy Process) originally developed in Pittsburgh by Prof. Thomas Saaty. It is good and you can certainly try it for free. However for “production” purposes it will cost you (or your company) thousands of dollars a year.

Of course, the calculations can all be done “by hand” but, truly, who would have time for that? You’re looking for a tool after all to save time.

This small university developed a simple but very useful Excel add-in that can do just that. Courtesy of Prof. Jaroslav Ramik who was behind the project it can be downloaded here. The Tool is called DAME (Decision Analysis Module for Excel) and this article is aiming at showing some of its very useful features. In other words by using it, you save not only time but money too.

So how does it work? The best start would be to consider a simple comparison of 4 products that was part of one of my previous articles. The products have the following prices:

A = 190B=230
C = 320D=290

It is the simplest evaluation task possible but even in this case hand calculations would still be quite time-consuming. You’re going to get exactly the same results (a final rank of evaluated alternatives) with DAME without using any calculations whatsoever in a matter of minutes.

All in all, DAME is an extremely neat solution.

It can be used for much more advanced tasks like requirements prioritization.

02 – Theory behind the AHP method (Analytic Hierarchy Process)

Analytic Hierarchy Process can be useful as a decision-support method in project management in instances where a few options (be it requirements, risks or other “alternatives” need to be prioritized or selected. Let’s take a look at some of the key theory concepts behind it.

There are two fundamental principles used in the general decision-making theory: that of deduction and that of induction (sometimes referred to as a system approach).  Deduction arrives at particulars from the general by applying logic. In other words it goes top-down, narrowing down more general truths into detailed conclusions.

The system approach on the other hand, is based on the premise that particulars are not as important. You are going bottom-up, as if trying to seize general truth.

The essence of the Analytic Hierarchy Process is based on employing both approaches. It is a method that first decomposes a complex problem in single components, and the components (sometimes referred to as variables), are put in a form of a hierarchy. They are then given numerical values. Each variable gets a value according to its importance in relation to other variables. (That is depends on whether it is quantitative or qualitative.) What follows is a synthesis of the values which will determine what weight each variable has in influencing the overall evaluation of the problem. Eventually all evaluated outcomes (alternatives/variants) will receive its total numerical value to form a ranking.

Many decision-making situations entail both physical as well as psychological aspect. Physical aspect can be regarded as objective, as it is from “tangible” realm, something that can be taken hold of or at least measured. Price as a criterion, for example can be quantified by money units, size or distance is quantifiable by units of measurement etc. On the other hand, the psychological aspect of the decision-making problem is more tricky. It is “intengible” in essence and there is no scale or range that might sufficiently, universally and unambiguously express it. They are often a product of subjective ideas, gut-feeling  or assumptions of an individual, a group or the whole society. Let’s take design qualities of a product as an example. The AHP deals with both of those aspects and is able to incorporate them as equal inputs of a unique decision-making support system.

Breaking down a seemingly complex problem into a clear hierarchy and only then focusing on different aspects of the decision, substantially expands possibilities of those who make the decision.

Analytic Hierarchy Process has been developed in Pittsburgh, USA in 1984, originally by Dr. Thomas L. Saaty – internationally recognized scholar and innovator of the decision-making theory.  It has since become one of the most successful and widely used decision support systems of today. It has grown into a comprehensive software tool used in collaborative, teamwork corporate decision-making.

Stages of AHP Using Expert Choice Software

As mentioned above, the largest contribution of AHP is its support in decision-making process by employing both subjective and objective factors when it comes to evaluating different alternatives of outcomes. Unlike in other methods, both quantitative data (that are clearly represented by numbers) and qualitative data (that often regarded as subjective) can be fed into the process. They are then assessed depending on importance that the decision maker has given to them but also in what layer of hierarchy they were placed. In a few steps, apparently advanced and complex decision-making problems can be solved in a relatively simple way.

The whole process can be divided into 5 stages:

1. Breaking down the problem into a hierarchy (analysis)
2. Evaluating criteria and decision alternatives on different levels of the hierarchy (setting priorities)
3. Measuring consistency of evaluation (finding consistency ratio)
4. Synthesis – generating overall weight for each evaluated decision alternative and their ranking
5. Sensitivity analysis

Breaking Down the Problem Into a Hierarchy (Analysis)

Decomposing the problem into a hierarchy is the first basic step of the Analytic Hierarchy Process. A hierarchy means a system of several levels, each including a finite number of elements. There is a mutual relationship between each two vertically-neighbouring levels. The higher the level is, the more general role it plays. Elements placed higher in the hierarchy controlled and managed by elements immediately underneath them. The element at the very top of the hierarchy is always the Goal of the decision-making process. The Goal has a weight that equals 1. 1 is than divided among the elements of the second level of the hierarchy, evaluation of elements in the second level of the hierarchy are then “dissolved” into the third level etc.

Hierarchy chosen depends on the character of the decision-making problem. There are a few types of the hierarchy:

  • Goal – Criteria – Alternatives
  • Goal – Criteria – Subcriteria – Alternatives
  • Goal – Experts – Criteria – Alternatives
  • Goal – Criteria – Intensity Levels – Multiple Alternatives

In most decision-making problems we will make do with the first mentioned type of hierarchy, that is Goal – Criteria – Alternatives but it can always be extended.

Evaluating Criteria and Decision Alternatives on Hierarchy Levels (Setting Priorities)

To set priorities for individual elements of the hierarchy, one must first know whether the data has a quantitative or qualitative nature. If quantitative, they may be ruled either by maximizing or minimizing. The maximizing rule will regard the highest value to be the best, while minimizing rule will regard the lowest value to be the best.

Evaluating by Quantitative Criteria

Price is a kind of a quantitative criterion that typically has a minimizing ruling when considered from the consumer’s point of view. The less it costs, the better. Let’s evaluate 4 products by price: A, B, C, and D, with the goal of assigning each one a numeric weight. The prices are:

Product            Price

A                190B 230
C                320D 290

Because the criterion is minimizing (lower values are considered better), the first step to calculate the weights is converting the values by using the following formula to get a coefficient kj:

kj=1Price100

This coefficient actually converts a minimizing criterion into a maximizing one (the higher the better because with price it’s the other way round):

Product kj
A 0.526
B 0.436
C 0.313
D 0.349

The resulting quantitative pj weights are calculated by a normalization formula:

\begin{equation}p_{j}=\frac{k_{j}}{\sum_k_{j}}\end{equation}

Product kj pj
A 0.526 0.324
B 0.436 0.268
C 0.313 0.193
D 0.349 0.215



Evaluating by Qualitative Criteria – Pairwise Comparisons

Pairwise comparisons belong to one of the most basic concepts of Analytical Hierarchy  Process. It is for evaluation of the criteria that are not clearly quantifiable but in the overall decision-making process play a crucial part. It is very difficult to assign weights to qualitative assessments by guessing and intuition, the AHP derives the information from comparing all the alternatives among themselves on every level of the hierarchy. In other words it slices the overall information into pairs of information. It is then used as a base for calculating numerical weights of each alternative.
Each pair of criteria being compared is assessed by 9-degree numerical scale that was developed specifically for this purpose:

Numeric Scale Description Explanation
1 Equal Both elements having same importance
3 Moderate Moderate importance of one over another
5 Strong Strong/essential importance of one over another
7 Very strong Very strong or demonstrated importance
9 Extreme Extreme importance of one over another

Besides the ones mentioned, there are also half-grades of 2, 4, 6, 8.

The relationship between elements in pairwise comparisons is called ‘importance’ of one over another but it can just as well be referred to as ‘preference’ or ‘likelihood’ of their occurrence. It always depends on the type of the problem being solved.

After pairwise comparisons of k-number of elements, pairwise comparison matrix (also known as Saaty’s matrix) is constructed. It is basically a reciprocal matrix consisting of k2 elements with 1’s on its diagonal and inverted values on each side. Typical pairwise comparison of 3 evaluated “Options” by a qualitative criterion can look as follows:

Option A Option B Option C
Option A 1 2 8
Option B ½ 1 6
Option C 1

For better clarity usually only the values in bold are shown. The number of those values can be calculated by the following formula:

n(n1)2

While AHP appears to be rather straightforward at first sight, the background mathwork needed for calculating numerical weights out of pairwise comparisons is not as straightforward. Eigenvalues and eigenvectors are involved and computer software such as Expert Choice does the hard work.

There is however an approximation method – an algorithm that can calculate rough weights in three steps without using a computer.

Algorithm for Calculating Approximate Weights

Step 1: Add up values in each column of pairwise comparison matrix

Option A Option B Option C
Option A 1 2 8
Option B ½ 1 6
Option C  ⅙ 1
Total 13/8 19/6 15


Step 2: Each item is divided by the total of its column thus getting a normalized matrix

Option A Option B Option C
Option A 8/13 12/19 8/15
Option B 4/13 6/19 6/15
Option C 1/13 1/19 1/15
Total 1 1 1



Step 3: Total of each row will be divided by the number of items in the row

Option A Option B Option C
Option A (8/13 + 12/19 + 8/15) / 3
Option B (4/13 + 6/19 + 6/15) / 3
Option C (1/13 + 1/19 + (1/15) / 3
Total 13/8 19/6 15

The calculated means will then serve as the approximate weights of each alternative (called Options here). From the numbers below we can see that Option A “won”.

Approximate weight

Option A        0.593

Option B        0.341

Option C        0.066

Total            1

Measuring Consistency of the Evaluation

Consistency of the measurements is way of expressing a certain ‘compactness’ of the preferences created during pairwise comparisons. It shows to what extent the data fed into the computer is logically cohesive. If for example, Alternative 1 is twice as important as Alternative 2 and Alternative 2 is three times as important as Alternative 3, then Alternative 1 must be 6 times (2 x3) as important as Alternative 3. This would be a case of a perfect consistency, in other words, the consistency coefficient would equal 0.

Inconsistency of judgements is at the background of human thinking. Humans don’t just use logic when drawing conclusions but rather stick to intuition, emotions, experience that all influence their attitudes and swing their decisions. If someone prefers apples to oranges and at the same time the person likes oranges better than bananas, shouldn’t it automatically be assumed that apples will be preferred over bananas? And yet the same people will still go for bananas rather than apples because there are other things to consider, like say, time of the day, season, etc. all eventually causing that they illogically, or ‘inconsistently’ choose this alternative and not the other.

In practical applications perfect consistency is rare because new and new information is constantly added in evaluation and it changes the previous relationships. Therefore pairwise comparisons permit a certain amount of inconsistency of preferences. The AHP works with the so called Consistency Ratio, with the rule of thumb that, if the inconsistency be more than 10 per cent, the evaluation should be revisited.

High inconsistency (bad consistency ratio) implies one of the following problems:

  • Ill logic in pairwise comparisons
  • Badly structured hierarchy
  • Errors/typos during data input

Expert Choice calculates consistency ratios automatically with each pairwise comparison. In case that the inconsistency is too high, it even has a feature that discloses the elements where inconsistency is highest. It could be repeated until inconsistency goes back to an acceptable level.

For illustration let’s look at how approximate inconsistency can be calculated without using a computer. The previous example is used:

Option A Option B Option C
Option A 1 2 8
Option B ½ 1 6
Option C 1

If Option A is preferred twice over Option B and Option B is preferred 6 times over Option C, then Option A should be preferred 12 times over Option C. That would be an ideal situation or perfectly consistent evaluation. However, because Option A is only 8 times more preferred than Option C, consistency ratio needs to be found.
Working out the approximate consistency ratio is dealt with in the next chapter.

Algorithm of Calculating Approximate Consistency Ratio

Step 1: Each column element of the original pairwise comparison matrix is multiplied by the resulting weight of their alternative and then rows are summarised:

Option A Option B Option C
(0.593) (0.341) (0.066)
Option A 1 2 8
Option B 0.5 1 6
Option C 0.125 0.167 1

will become

Option A Option B Option C
Option A 0.593 0.682 0.528
Option B 0.297 0.341 0.396
Option C 0.074 0.057 0.066

Resulting totals then are:

Option A        1.803

Option B        1.034

Option C         0.197

Step 2: Each resulting total is divided by its weight:

Option A        1.803 / 0.593 = 3.04

Option B        1.034 / 0.341 = 3.032

Option C         0.197 / 0.066 = 2.985

Step 3: Mean is calculated:
Lmax=3.04+3.032+2.9853=3.019

Step 4: Consistency Index CI=Lmaxnn1 is then calculated, where is number of elements being compared:

CI=3.01932=0.0095

Step 5: Consistency ratio is calculated using the so called Random Index – which is an average consistency index of a randomly generated n x n – size matrix:

CR=CIRI

Random Index (RI) doesn’t need to be calculated as it is already provided in the following chart:

n        RI
2        0
3        0.58
4        0.9
5        1.12
6        1.24
7        1.32
8        1.41

For n = 3 RI is 0.58 and the Consistency Ratio is:

CR=0.00950.58=0.016

The approximate Consistency Ratio of the three evaluated Options equals 0.016 and meets the previously stated expectation of CR < 0.1.

The evaluations are therefore considered to be sufficiently consistent.

Practical Application of Expert Choice

Analytical Hierarchy Process and Expert Choice has a wide application in real-life business-related situations. Generally, it has large usage in marketing in product comparisons but it can just as well be used to support decision-making or in planning, investing, conflict resolution, forecasting or risk management to name a few.
IBM used Expert Choice when applying for Malcolm Baldridge National Quality Award. General Motors used it in its design projects when evaluating prototypes of its new products. Xerox used it for the portfolio management, evaluating new technologies and as a support tool in marketing decisions. It has been used by government in rating of buildings by historic significance, or in assessing the condition of highways so the engineers could determine the optimum scope of the project and justify the budget to lawmakers.

In project management it can  be used in the scope management knowledge area, in estimating cost of work packages through control account level, then aggregating them into the overall project cost estimates.
It has a large usage in Human Resources in Acquire Team process to evaluate employees or potential team members from large number of applicants against the set of defined criteria. They are quickly rated and scored to select the ones that best suit the criteria.

Portfolio management is another ara of application where it helps decision-makers rate the business value of their potential projects. AOL project portfolio management can be an example.
AHP can also be used in risk management, identification and prioritization where both subjective inputs (qualitative risk analysis) and quantitative data (quantitative risk analysis) need to be assessed.

03 – Selecting Seller by Source Selection Criteria using DAME

In procurement management it is often necessary to evaluate sellers based on proposals they have sent. Evaluation is crucial for the award decision.

Proposal evaluation is an assessment of the proposal and the offeror’s ability to perform the prospective contract successfully. Here is a simplified example how this process can be done with a little known tool called DAME.

Let’s say you’re evaluating 4 Sellers according to 4 evaluation criteria, which are:

Bid ($) – money asked to carry out the job -> “MIN” (more about “minimizing” and “maximazing” here)

Past Performance – Excellent (2), Satisfactory (1.5), N/A (1), Bad (0.5) -> “MAX”

Know how possessed by sellersPairwise comparison (we’re comparing sellers against one another) -> PAIRWISE COMPARISON

Own resources available – Pairwise comparison (we’re comparing sellers against one another) -> PAIRWISE COMPARISON

There are 2 key authorities submitting their judgements. That means 2 “scenarios”. One of them will be the CEO and the other will be the PM (Project Manager).

Download the whole example from the link below.
Download
Selecting Seller by Source Selection Criteria using DAME

04 – Requirement Prioritisation using DAME

Sometimes little means more. This little-known and free Excel add-in can do exactly what many powerful propriatary software packages. DAME stands for Decision Analysis Module for Excel.

If you haven’t worked with DAME before, here is a short instruction manual.
Why don’t you make your life easier. Using this little-known free Excel add-in you can save thousands. DAME stands for Decision Analysis Module for Excel.

Let’s say you lead a development team and you prioritize the development features. Based on the Pareto principle you know that 80% of of effort will be spent on only 20% of features. In other words you need to spend the bulk of your money on one fifth of the most important features. You narrow the feature list down to only say 10 most important ones – those that are an absolute Must Have for the customer.  However you need a better picture when it comes to prioritizing those 10 key requirements.  Evaluations is being carried out by 5 experts including a customer representative. Their estimates must be synthesized into a final outcome to have a better idea how the budget should be shaped for the next ‘iteration’, sprint, run, or whatever agile terminology is used at your organisation.

One way of doing this could be by using DAME.

Let’s say, with a certain amount of simplification, that you already have a complete and up-to-date requirements breakdown list with say 12 categories of user requirements. The requirements are coded by: a number –  representing the category and a letter – standing for the actual requirement.  E.g. 1a, 4c, etc.

You know that the 10 most important requirements for the moment are:
1=> 1a – users able to record, view, edit info of all clients who have ever entered service
2=> 1b – records will be modeled on a used government reporting tool
3=> 3a – users can record clinic activity (prerequisite in applying for gov grants)
4=> 3b – clinic info will be categorized
5=> 5a – information is confidential (storing on secure server, only for the righ eyes)
6=> 6a – communications book ready with no editing allowed (24/7 shifts need to pass critical information)
7=> 8a – client exclusion list – a list of clients currently banned from service available from anywhere to everybody
8=> 10b – assigned staff will be able to edit and delete client information
9=> 11d – Client will have a special ID that matches the currently used government reporting tool
10=> 11f – One database will be shared by two physical facilities (on different physical addresses). Each organisation will only see data related to it.

Now let’s assume you think that those requirements should be seen as follows (1 = most important; 10 = least important):
Order; Requirement Code; Rank

1. 11d
2. 5a
3. 8a
4. 3a
5. 6a
6. 11f
7. 10b
8. 1a
9. 3b
10. 1b

That’s your evaluation. However there are 4 more stakeholders involved and they might see the whole thing differently.

This problem (when being resolved in DAME) calls for the following input:
10 variants (10 requirements being evaluated)
5 scenarios (5 decision-makers or evaluators)
1 criterion (highest likelyhood that the customer will be happy if a particular requirment is met by the end of the next development cycle; it is a kind of an expert judgement) The important thing here is that is is a minimizing criterion because (you rank the requirements by your judgement from 1 to 10 where 1 is most important and 10 is least important – that means – the less the better)

You decide that the weights (decision-making power) of the stakeholders (based on their position) is as follows:

Customer Rep = 50%
Project Mngr = 20%
Developer = 20%
Tester = 5%
Coordinator 5%

The results from the 5 decision-making parties are then synthesized into one final outcome for each requirement. This way you receive the final ranks for your ten most important requirements:

1. 11f = 0.206557377
2. 11d = 0.133911394
3. 10b = 0.133518493
4. 3b = 0.105744479
5. 5a = 0.093889717
6. 1a = 0.059748002
7. 8a = 0.056868988
8. 1b = 0.05641512
9. 6a = 0.053082238
10. 3a = 0.050264192

If your budget is 100,000yousplititagainsttheweightstoseehowmuchmoneyshouldgotowardsmeetingeachrequirment:1.11f=0.206557377>20,656
2. 11d = 0.133911394 -> 13,3913.10b=0.133518493>13,352
4. 3b = 0.105744479 -> 10,5745.5a=0.093889717>9,389
6. 1a = 0.059748002 -> 5,9757.8a=0.056868988>5,687
8. 1b = 0.05641512 -> 5,6429.6a=0.053082238>5,308
10. 3a = 0.050264192 -> $5,026

The above example is simplified of course for demonstration purposes only it is only to show that using tools such as DAME is elegant, quick and neat, and most importantly, it doesn’t cost anything.

Optimizing Ontario’s investments in a “basket” of core mental health services for children and youth – background

Some background

The Ministry of Children and Youth Services (MCYS) in Ontario funds service providers to deliver community-based child and youth mental health (CYMH) services under the authority of the Child and Family Services Act, R.S.O. 1990, c.C.11 (CFSA). The paramount purpose of the CFSA is to promote the best interests, protection and well-being of children.

Some terms

Client

The MCYS defines a client  as “the intended recipient of the child and youth mental health core service.” The client is a child or youth under 18 years of age who is experiencing mental health problems. In addition to mental health needs, clients may also be experiencing additional challenges related to their development or have specific impairments and/or diagnoses, including a developmental disability, autism spectrum disorder or substance use disorder. Other conditions or diagnoses do not preclude clients from receiving mental health services, but may add to the complexity of their needs, and the services they require. Similarly, where children and youth are involved in other sectors (e.g. youth justice and child welfare) these circumstances do not preclude them from receiving core services. Where children and youth have additional needs and are receiving a range of services, the focus must be on how the services connect. A coordinated approach to service delivery must be supported. Families (including parents, caregivers, guardians, siblings and other family members) may also receive services from a core service provider, in order to address the identified needs of the child or youth client. This may occur when the participation in treatment is recommended to support the child or youth’s service plan.

Continuum of needs-based mental health services

Children, youth and their families can benefit from access to a flexible continuum of timely and appropriate mental health services and supports, within their own cultural, environmental and community context. Mental health promotion, prevention, and the provision of services to address mental health problems represent different points along the continuum. Children, youth and their families may enter the continuum of needs-based services and supports at any point. The actual services a child/youth needs will vary. For example, some children/youth may benefit from targeted prevention services, while others will require more specialized mental health services. In addition, a child or youth’s mental health service needs may change over the course of their treatment.

The following schematic outlines the full continuum of needs-based mental health services and supports, and shows how core services fit within this continuum. It also represents the relative demand for services – level one reflects all children and youth, while level four focuses on a smaller subset of the child/youth population with the most severe, complex needs. This schematic is for service planning only – it is not used for diagnosis or for determining the appropriateness of specific mental health interventions.

MCYS - Continuum of core mental health services
Continuum of CYMH Needs-Based Services and Supports. *Includes members of a group that share a significant risk factor for a mental health problem(s).

Service areas

After a thorough review – including an assessment of Statistics Canada’s census divisions – the MCYS has identified 34 service areas in Ontario for the purpose of:

  • ensuring that all clients across the province will be able to access the same core services
  • facilitating planning, and
  • creating pathways to care.

The defined service areas are not barriers to service. Clients will be able to access service from any service area.

Core services

The MCYS has defined a set of core children and youth mental health services (“core services”) to be available within every service area and has established minimum expectations for how core services are planned, delivered and evaluated. Core services may not be available in every service area immediately – the expectation is that they will be made available over time as lead agencies assume their roles and responsibilities.

Core services represent the range of MCYS-funded CYMH services that lead agencies are responsible for planning and delivering across the continuum of mental health needs within each service area. It is recognized that children and youth in receipt of core mental health services may also require other services and supports. For example, children and youth may receive more than one core service as part of a service plan, as well as other services funded by MCYS or other partners.

Seven core services are to be available across all service areas:

  • Targeted Prevention
  • Brief Services
  • Counselling and Therapy
  • Family Capacity Building and Support
  • Specialized Consultation and Assessments
  • Crisis Support Services, and
  • Intensive Treatment Services.

The MCYS funds providers of core services through the following detail codes:

  • A348 – Brief Services
  • A349 – Counselling and Therapy
  • A350 – Crisis Support Services
  • A351 – Family/Caregiver Capacity Building and Support
  • A352 – Coordinated Access and Intake
  • A353 – Intensive Treatment Services
  • A354 – Case Management and Service Coordination
  • A355 – Specialized Consultation and Assessment
  • A356 – Targeted Prevention Term

The MCYS has identified a target population for each core service. This is the population for whom the service is designed, and for whom the service is intended to provide better mental health outcomes. The act of defining a target population is not meant to be exclusionary. Rather, it is a means to support planning and delivery in a way that benefits the children and youth who are in greatest need of the mental health service. In general, the target population for core services includes those children and youth under 18 years of age and their families who are experiencing mental health problems along levels two, three and four of the CYMH continuum. Additional target populations may also be identified within specific core services.

Lead agency

In every service area, the MCYS has identified a lead agency that will be responsible for the planning and delivery of high-quality core services across the continuum of mental health services in the service area.

A lead agency may either directly deliver core services or work with other providers of core services to deliver the full range of core services within the service area. Lead agencies are responsible for engaging cross-sectoral partners in the health and education sectors, including the relevant Local Health Integration Network (LHIN) and school boards. Lead agencies will connect with other providers to plan and enhance mental health service pathways for children and youth and improve transparency, so that everyone will know what to expect.

Providers of core services are required to comply with the Program Guidelines and Requirements #01 (PGR #01): Core Services and Key Processes.

The core services, key processes and functioning of the CYMH service sector will require refinement from time to time as other provincial initiatives and activities are developed and implemented. Within the broader context of these new initiatives, it is important that the roles and responsibilities of all core service providers are made clear and that the linkages between these services are transparent.

Planning to transform child and youth mental health services in Ontario

A key driver of Moving on Mental Health is the need to build a system in which children, youth and their families:

  • Have access to a clearly defined set of core child and youth mental health services
  • Know what services are available in their communities and how they are connected to one another, and
  • Have confidence in the quality of care and treatment. In a mature system, one of the ways in which this vision will be realized is through identification of lead agencies with planning and funding accountability for core child and youth mental health services within defined service areas.

Within each defined service area, the lead agency will be responsible for:

  • Delivering and/or contracting for the range of defined core CYMH services
  • Making them accessible to parents, youth, and children, and

Establishing inter-agency and inter-sectoral partnerships, protocols and transparent pathways to care. These responsibilities fall into two broad categories:

  • Core Service responsibilities – which relate to the defined core services delivered by the community-based child and youth mental health sector, as well as the key processes that enable high-quality service, and
  • Local System responsibilities – which relate to the collaboration of the community-based sector with other parts of the service continuum such as those supports and services delivered by health care providers, schools and others.

The Core Services Delivery Plan and the Community Mental Health Plan for children and youth will set out how the lead agency carries out these responsibilities. The lead agency will be responsible for developing these plans, and is expected to work collaboratively with other mental health service providers and with all sectors that support children and youth and respond to their mental health needs.

The Core Services Delivery Plan will, together with the Community Mental Health Plan, provide critical insight into each service area and guide activities as we move forward with transforming the experience of children, youth and families. The intent is that over time, both of these plans will have a three-year horizon and will be updated annually, since they inform one another. They will also provide content for the Accountability Agreement entered into between the lead agency and MCYS.

Core Services Delivery Plan

The development of a Core Services Delivery Plan (CSDP) is a key planning and communication tool that will document expectations, obligations and commitments for the provision of core services and associated key processes in each defined service area. This reflects the need to establish a consistent approach that will support critical insights into local and provincial child and youth mental health service issues, while recognizing the unique circumstances of lead agencies and service areas. The Core Services Delivery Plan documents how core services will be delivered in the defined service area. It consists of three areas of content:

  • Service Commitments
  • Continuous Improvement Priorities, and
  • Budget.

In developing the plan, the lead agency and child and youth mental health service providers should ask themselves some key questions:

  • Can we demonstrate that the full range of core services is available in our service area, and that minimum expectations set out in the Service Framework are being met?
  • Can we show how our services are getting better at meeting the mental health needs of children and youth in our communities?
  • Are we making the best possible use of limited resources to deliver high-quality services?
A. Service Commitments

This section of the plan will:

  • Identify, with specific activities and time frames:
    • How the lead agency and other child and youth mental health service providers in the service area will address the expectations set out in the Service Framework, including who will deliver what services over the projected three-year time horizon
    • Where changes to services or service providers are proposed, the plan will document how the changes will result in improvement to child and youth mental health outcomes, service quality and efficiencies
  • Indicate how, if a change in service providers or in contracted relationships is proposed, it will be handled in a transparent manner with due regard to minimizing disruption to service
  • Set out how services will be designed and delivered in a culturally responsive manner to address diverse populations including francophone and Aboriginal populations
  • Document how a clear, stable point of contact for children and youth with mental health needs and their families, as well as those seeking services on their behalf, will be established and/or maintained
  • Report on the reach and efficacy of programs and services, including how input from parents and youth has been incorporated to ensure that what has been developed works for them, and
  • Describe the process by which the lead agency has engaged and will continue to engage respectfully with all core child and youth mental health service providers in the service area.
B. Continuous Improvement Priorities

This section of the plan will:

  • Monitor and report on the impact of current programs and services
  • Identify improvement priorities, taking into account priorities established by MCYS and the expectations set out in the Service Framework, in areas such as service quality and outcomes, a purposeful approach to wait list and wait time management, and others over the three-year horizon of the plan
  • Set out specific activities and time frames that will support continuous improvement goals and priorities, and
  • Address matters such as data sharing protocols between the lead agency and other child and youth mental health agencies in the service area, that will support monitoring and reporting on performance indicators in order to enable tracking of trends, challenges and opportunities for continuous improvement.
C. Budget

This section of the plan will:

  • Forecast activities, resource allocations and budget over the three-year horizon, including financial implications of planned changes to service delivery.

Community Mental Health Plan for children and youth

System responsibilities are built on key partnerships and collaborations developed at the local level to support young people and their families across the full continuum of needs. Although service areas may differ in terms of their service profile, service patterns, as well as the degree of pre-existing cooperation and collaboration across systems and sectors, the lead agency will be responsible for bringing partners together to create coherence for children, youth and their families. MCYS is working, together with the Ministry of Health and Long-Term Care and the Ministry of Education, to put in place conditions that will support this important work.

The Community Mental Health Plan for children and youth will be a public document that is developed by the lead agency and describes the processes by which:

The lead agency has engaged and will continue to engage respectfully with sector partners such as organizations funded by Local Health Integration Networks, District School Boards, public health units, hospitals, primary health care providers, and those delivering MCYS-funded services (e.g., child welfare, autism services) and others, and

Input from parents and youth has been incorporated to ensure that what has been developed works for them. It will cover the following topic areas:

  • Understanding current needs and services
  • Collaborative planning, and
  • Pathways to, through and out of care.

In developing the plan, the lead agency, child and youth mental health service providers and partners from all sectors involved with child and youth mental health should ask themselves some key questions:

  • Are all those who serve children and youth working together systematically to address mental health needs in the service area?
  • Are the roles and responsibilities of everyone across the continuum of needs and services clear to parents, youth and those seeking services on their behalf, including how services are accessed?
  • Are there shared commitments to address service gaps and expand on opportunities to better meet identified needs?
A. Understanding current needs and services
  • Report on a needs assessment of the current state of child and youth mental health services across the service area, identifying gaps and opportunities for meeting needs across the continuum, and
  • Identify and maintain an inventory of who is providing which services to meet the needs identified.
B. Collaborative planning
  • Establish mechanisms to explore, on an ongoing basis, opportunities to leverage resources, reduce duplication, enhance outcomes, and create added value for children and youth with mental health needs through collaboration and joint planning, and
  • Identify and document commitments and actions to be taken to address shared and agreed upon priorities, together with associated timelines and measures to assess results.
C. Pathways to, through and out of care

Develop and document protocols, processes and partnerships that exist, or will be developed, that will streamline and strengthen clear pathways to, through and from care across sectors.

Next: Applying Multi-Criteria Decision Analysis to the “basket” of core mental health services for children and youth in Ontario

MCYS – Child and Youth Mental Health in Ontario – Resources

Service Framework

Program Guidelines

 

An invitation to Portfolio Decision Analysis

Source: Salo, A, Keisler, J and Morton, A – An invitation to Portfolio Decision Analysis – Ch 1 in Portfolio Decision Analysis: Improved methods for resource allocation (2011).

Organizations and individuals have goals that they seek to attain by allocating resources to actions that consume resources. These scenarios involve one or several decision makers who are faced with alternative courses of action which, if implemented, consume resources and enable consequences.The availability of resources is typically limited by constraints while the desirability of consequences depends on preferences concerning the attainment of multiple objectives. Furthermore, the decision may affect several stakeholders who are impacted by the decision even if they are not responsible for it. There can be uncertainties as well, for instance, at the time of decision making, it may be impossible to determine what consequences the actions will lead to or how much resources they will consume.

These, in short, are the key concepts that characterize decision contexts where the aim is to select a subset consisting of several actions with the aim of contributing to the realization of consequences that are aligned with the decision maker’s preferences.

Portfolio Decision Analysis (PDA)
A body of theory, methods, and practice which seeks to help decision makers make informed multiple selections from a discrete set of alternatives through mathematical modeling that accounts for relevant constraints, preferences, and uncertainties.

PDA differs from the standard decision analysis paradigm in its focus on portfolio choice as opposed to the choice of a single alternative from a set. There are analytical arguments as to why the pooling of several single choice problems into a more encompassing portfolio choice problem can be beneficial.

  1. The solution to the portfolio problem will be at least as good, because the combination of single choice problems, when considered together, constitutes a portfolio problem where there is a constraint to choose one alternative from each single choice problem. Thus, when considering these single choice problems together, the removal of these (possibly redundant) single choice constraints may lead to a better solution.
  2. If the single choice problems are interconnected – for instance, due to the consumption of shared resources or interactions among alternatives in different subsets – the portfolio frame may provide a more realistic problem representation and consequently better decision recommendations.

A key question in PDA is therefore what alternatives can be meaningfully analyzed as belonging to the “same” portfolio. While PDA methods do not impose inherent constraints on what alternatives can be analyzed together, there are nevertheless considerations which suggest that some alternatives can be more meaningfully treated as a portfolio:

  • when the alternatives consume resources from the same shared pool
  • when the alternatives are of the same “size” (measured, e.g. in terms of cost, or the characteristics of anticipated consequences)
  • when the future performance of alternatives is contingent on decisions about what other alternatives are selected, or
  • when the considerationof alternatives together as part of the same portfolioseems justified by shared responsibilities in organizational decision making.

The fact that there are more alternatives in portfolio choice suggests also that stakes may be higher than in single choice problems. Thus, the adoption of a systematic PDA approach may lead to particularly substantial improvements in the attainment of desired consequences.

But apart from the actual decision recommendations, there are even other rationales that can be put forth in favor of PDA-assisted decision processes. For example, PDA enhances the transparency of decision making, because the structure of the decision process can be communicated to stakeholders and the process leaves an auditable trail of the evaluation of alternatives with regard to the relevant criteria. This, in turn, is likely to enhance the efficiency of later implementation phases and the accountability of decision makers.

Evolution of Portfolio Decision Analysis (FYI)

  • financial portfolio optimization
  • capital budgeting models
  • quantitative models for project selection
  • decision analysis
  • from decision analysis to portfolio decision analysis

Embedding PDA in organizational decision making (FYI)

  • embedding PDA in organizational decision making
  • extending PDA theory, methods and tools
  • expanding the PDA knowledge base

Moglen, Eben – Innovation under Austerity – 20120522

Moglen, Eben – Innovation under Austerity – 20120522

Video.

Transcription by Ben Asselstine.

Thank you. it’s a pleasure to be here, and to see so many friends. I’m very grateful to David for the invitation, it’s a privilege to be here. I’m going to talk of mostly about a subject almost as geeky as stuff we all talk about all the time, namely political economy. I’m going to try and make it less snooze-worthy than it sometimes seems to be, but you’ll forgive me I’m sure for starting fairly far from OpenSSL, and we’ll get closer as time goes by.

The developed economies around the world, all of them now, are beginning to experience a fundamentally similar and very depressing condition. They are required to impose austerity because levels of private debt have gummed up the works and the determination of the owners of capital to take vast risks with other peoples money have worked out extremely badly for the last half decade. And so austerity is the inevitable and politically damaging position for all the governments in the developed world, and some of those governments have begun to slip into a death spiral, in which the need to impose austerity and reduce public investment and welfare support for the young is harming economic growth, which prevents the austerity from having its desirable consequences. Instead of bad asset values being worked off and growth resuming, we are watching as the third largest economy in the world, the European Union, finds itself at the very verge of a currency collapse and a lost generation, which would have a profoundly depressing effect on the entire global economy.

For the policy-makers–I recognize that few of them are here, they have of course, better things to do than to listen to us–for the policy-makers in other words, an overwhelming problem is now at hand, how do we have innovation and economic growth under austerity?

They do not know the answer to this question and it is becoming so urgent that it is beginning to deteriorate their political control. Marginal parties in several very highly developed and thoughtful societies are beginning to attract substantial numbers of votes, and threatening the very stability of the economic planners’ capacity to solve, or to attempt to solve, the problem of innovation under austerity.

This is not good news for anybody. This is not good for anybody. We have no opportunity to cheer for this outcome, which is largely the result of incompetence in those people who claim to be worth all that money because they’re so smart, it is partly the result of the political cowardice that gave them too much room to swing their cats. It is not that we are glad to see this happen, but there is a silver lining to the cloud. There are very few people who know how to have innovation under austerity. We are they.

We have produced innovation under austerity for the last generation and not only did we produce pretty good innovation we’ve produced innovation that all the other smart rich people took most of the credit for. Most of the growth that occurred during this wild and wacky period in which they took other people’s money and went to the racetrack with it, was with innovation we produced for them. So now, despite the really bad circumstances which we too can deplore, because the unemployment is my graduating law students, your children, and all those other young people whose lives are being harmed for good by current bad economic circumstances. The people beginning their careers now will suffer substantial wage losses throughout their lifetimes. Their children will get a less good start in life because of what is happening now, we cannot be pleased about this.

But we have a very substantial political opportunity. Because we do have the answer to the most important question pushing all the policy-makers in the developed world right now. That means we have something very important to say and I came here this morning primarily to begin the discussion about precisely how we should say it.

And I want to present a working first draft of our argument, I say “our” because I look around the room and I see it’s us here this morning. Our argument, about what to do with the quandary the world is in. Innovation under austerity is not produced by collecting lots of money and paying it to innovation intermediaries. One of the most important aspects of 21st century political economy is that the process we call disintermediation, when we’re begin jargony about it, is ruthless, consistent, and relentless. Television is melting. I don’t need to tell you that, you know already. Nobody will ever try to create a commercial encyclopedia again. Amazon’s lousy little I-will-let-you-read-some-books-unless-I-decide-to-take-them-back machine is transforming publishing by eliminating the selective power of the book publishers, much as Mr. Jobs almost destroyed the entire global music industry under the pretense of saving it. A task his ghost is already performing for the magazine publishers as you can see.

Disintermediation, the movement of power out of the middle of the net, is a crucial fact about 21st century political economy. It proves itself all the time. Somebody’s going to win a Nobel Prize in economics for describing in formal terms the nature of disintermediation. The intermediaries who did well during the past 10 years, are limited to two sets: health insurers in the United States owing to political pathology, and the financial industry.

Health insurers in the United States may be able to capitalize on the continuing political pathology to remain failing and expensive intermediaries for a while longer. But the financial industry crapped in it’s own nest and is shrinking now and will continue for some time to do so. The consequence of which is that throughout the economic system, as the policy-maker observes it, the reality that disintermediation happens and you can’t stop it becomes a guiding light in the formation of national industrial policy. So we need to say it’s true about innovation also.

The greatest technological innovation of the late 20th century is the thing we now call the World Wide Web. An invention less than 8000 days old. That invention is already transforming human society more rapidly than anything since the adoption of writing. We will see more of it. The nature of that process, that innovation, both fuels disintermediation, by allowing all sorts of human contacts to occur without intermediaries, buyers, sellers, agents, and controllers. And poses a platform in which a war over the depth and power of social control goes on, a subject I’ll come back to in a few minutes. For now what I want to call attention to is the crucial fact that the World Wide Web is itself a result of disintermediated innovation.

What Tim first did at CERN was not the Web as we know it now, the Web as we know it now was made by the disintermediated innovation of an enormous number of individual people. I look back on what I wrote about the future of personal homepages in 1995, and I see pretty much what I thought then would happen happening, I said then those few personal homepages are grass seed and a prairie is going to grow, and so it did.

Of course, like all other innovation there were unintended consequences. The browser made the Web very easy to read. Though we built Apache, though we built the browsers, though we built enormous numbers of things on top of Apache and the browsers, we did not make the Web easy to write. So a little thug in a hooded sweatshirt made the Web easy to write, and created a man in the middle attack on human civilization, which is unrolling now to an enormous music of social harm. But that’s the intermediary innovation that we should be concerned about. We made everything possible including, regrettably, PHP, and then intermediaries for innovation turned it into the horror that is Facebook. This will not turn out, as we can already see from the stock market result, to be a particularly favourable form of social innovation. It’s going to enrich a few people. The government of Abu Dhabi, a Russian thug with a billion dollars already, a guy who can’t wait to change his citizenship so he doesn’t have to pay taxes to support the public schools, and a few other relics of 20th century misbehavior.

But the reality of the story underneath is, if we’d had a little bit more disintermediated innovation, if we had made running your own web-server very easy, if we had explained to people from the very beginning how important the logs are, and why you shouldn’t let other people keep them for you, we would be in a rather different state right now. The next Facebook should never happen. It’s intermediated innovation serving the needs of financiers, not serving the needs of people. Which is not say that social networking shouldn’t happen, it shouldn’t happen with a man in the middle attack built into it. Everybody in this room knows that, the question is how do we teach everybody else.

But as important as I consider everybody else to be right now, I want to talk about the policy-makers: how do we explain to them? And here we begin to divide the conversation into two important parts. One, what do we know about how to achieve innovation under austerity? Two, what prevents governments from agreeing with us about that?

So let me present first my first draft of the positive case for innovation under austerity, it’s called “We Made The Cloud”.

Everybody understands this in this room too. The very point about what’s happening to information technology in the world right now, has to do with scaling up our late 20th century work. We created the idea that we could share operating systems and all the rest of the commoditizable stack on top of them. We did this using the curiosity of young people. That was the fuel, not venture capital. We had been at it for 15 years, and our stuff was already running everywhere, before venture capital or even industrial capital raised by IT giants came towards us. It came towards us not because innovation needed to happen, but because innovation had already happened, and they needed to monetize it. That was an extremely positive outcome, I have nothing bad to say about that. But the nature of that outcome, indeed the history as we lived it and as other can now study it, will show how innovation under austerity occurred. It’s all very well to say that it happened because we harnessed the curiosity of young people, that’s historically correct. But there’s more than that to say.

What we need to say is that that curiosity of young people could be harnessed because all of the computing devices in ordinary day to day use were hackable. And so young people could actually hack on what everybody used. That made it possible for innovation to occur, where it can occur, without friction, which is at the bottom of the pyramid of capital. This is happening now elsewhere in the world as it happened in the United States in the 1980s. Hundreds of thousands of young people around the world hacking on laptops. Hacking on servers. Hacking on general purpose hardware available to allow them to scratch their individual itches, technical, social, career, and just plain ludic itches. “I wanna do this it would be neat.” Which is the primary source of the innovation which drove all of the world’s great economic expansion in the last 10 years. All of it. Trillions of dollars of electronic commerce. Those of you old enough to remember when fighting Public Key Encry ption tooth and nail, was the United States government’s policy will remember how hard they fought, to prohibit 3.8 trillion dollars worth of electronic commerce from coming into existence in the world.

We were supposedly proponents of nuclear terrorism and pedophilia in the early 1990s, and all the money that they earned in campaign donations and private equity profits and all the rest of it, is owing to the globalization of commerce we made possible, with the technology they wanted to send our clients to jail for making. That demonstrates neatly I think, to the next generation of policy-makers how thoroughly their adherence to the received wisdom is likely to contribute to the death spiral they now fear they’re going to get into. And it should embolden us to point out once again that the way innovation really happens is that you provide young people with opportunities to create on an infrastructure which allows them to hack the real world, and share the results.

When Richard Stallman wrote the call at the university in Suffolk for the universal encyclopedia, when he and Jimmy Wales and I were all much younger than we are now, it was considered a frivolous idea. It has now transformed the life of every literate person in the world. And it will continue to do so.

The nature of the innovation established by Creative Commons, by the Free Software Movement, by Free Culture, which is reflected in the Web in the Wikipedia, in all the Free Software operating systems now running everything, even the insides of all those locked-down vampiric Apple things I see around the room. All of that innovation comes from the simple process of letting the kids play and getting out of the way. Which, you are aware, we are working as hard as we can to prevent now completely. Increasingly, all around the world the actual computing artifacts of daily life for human individual beings are being made so you can’t hack them. The computer science laboratory in every twelve year old’s pocket is being locked-down.

When we went through the anti-lockdown phase of the GPL 3 negotiations in the middle of last decade, it was somehow believed that the primary purpose for which Mr. Stallman and I were engaged in pressing everybody against lock-down had something to do with bootlegging movies. And we kept saying, this is not the Free Movie Foundation. We don’t care about that. We care about protecting people’s right to hack what they own. And the reason we care about it is, that if you prevent people from hacking on what they own themselves, you will destroy the engine of innovation from which everybody is profiting.

That’s still true. And it is more important now precisely because very few people thought we were right then, and didn’t exert themselves to support that point of view, and now you have Microsoft saying we won’t allow third-party browsers on ARM-based Windows RT devices. And you have the ghost of Mr. Jobs trying to figure out how to prevent even a free tool chain from existing in relation to IOS, and you have a world in which increasingly the goal of the network operators is to attach every young human being to a proprietary network platform with closed terminal equipment that she can’t learn from, can’t study, can’t understand, can’t whet her teeth on, can’t do anything with except send text messages that cost a million times more than they ought to.

And most of the so-called innovation in the world, in our sector, now goes into creating IT for network operators that improves no technology for users. Telecomms innovation in the world has basically ceased. And it will not revive so long as it is impossible to harness the forms of innovation that really work under austerity.

This has a second-order consequence of enormous importance. Innovation under austerity occurs in the first-order because the curiosity of young people is harnessed to the improvement of the actual circumstances of daily life. The second-order consequence is that the population becomes more educated.

Disintermediation is beginning to come to higher education in the United States, which means it is beginning to come to higher education around the world. We currently have two models. Coursera, is essentially the googlization of higher education, spun-off from Stanford as a for-profit entity, using closed software and proprietary educational resources. MITx, which has now edX through the formation of the coalition with Harvard University, is essentially the free world answer. Similar online scalable curriculum for higher education delivered over Free Software using free education resources. We have an enormous stake in the outcome of that competition. And it behooves all of us to put as much of our energy as we can behind the solutions which depend upon free courseware everybody can use, modify and redistribute, and educational materials based on the same political economy.

Every society currently trying to reclaim innovation for the purpose of restarting economic growth under conditions of austerity needs more education, deliverable more widely at lower cost, which shapes young minds more effectively to create new value in their societies. This will not be accomplished without precisely the forms of social learning we pioneered. We said from the beginning that Free Software is the world’s most advanced technical educational system. It allows anybody anywhere on earth, to get to the state of art in anything computers can be made to do, by reading what is fully available and by experimenting with it, and sharing the consequences freely. True computer science. Experimentation, hypothesis formation, more experimentation, more knowledge for the human race.

We needed to expand that into other areas of culture, and great heroes like Jimmy Wales and Larry Lessig laid out infrastructure for that to occur, we now need to get governments to understand how to push it further.

The Information Society Directorate of the European Commission issued a report 18 months ago, in which they said that they could scan 1/6th of all the books in European libraries for the cost of 100 km of roadway. That meant, and it is still true, that for the cost of 600 km of road, in an economy that builds thousands of kilometers of roadway every year, every book in all European libraries could be available to the entire human race, it should be done. [shout of “Copyright” from audience] Remember that most of those books are in the public domain, before you shout copyright at me. Remember that the bulk of what constitutes human learning was not made recently, before you shout the copyright at me. We should move to a world in which all knowledge previously available before this lifetime is universally available. If we don’t, we will stunt the innovation which permits further growth. That’s a social requirement. The copyright bargain is not immutable. It is merely convenient. We do not have to commit suicide culturally or intellectually in order to maintain a bargain which does not even relevantly apply to almost all of important human knowledge in most fields. Plato is not owned by anybody.

So here we are, asking ourselves what the educational systems of the 21st century will be like, and how they will socially distribute knowledge across the human race. I have a question for you. How many of the Einsteins who ever lived were allowed to learn physics? A couple. How many of the Shakespeares who ever lived, lived and died without learning to read and write? Almost all of them. With 7 billion people in the world right now, 3 billion of them are children; how many Einsteins do you want to throw away today? The universalization of access to education, to knowledge, is the single-most important force available for increasing innovation and human welfare on the planet. Nobody should be afraid to advocate for it because somebody might shout “copyright”.

So we are now looking at the second-order consequence of an understanding of how to conduct innovation under austerity. Expand access to the materials that create the ability to learn, adapt technology to permit the scientists below age 20 to conduct their experiments and share their results, permit the continuing growth of the information technology universe we created, by sharing, over the last quarter century, and we’d begin to experience something like the higher rates of innovation available, despite massive decreases in social investment occurring because of austerity.

We also afford young people an opportunity to take their economic and professional destinies more into their own hands, an absolute requirement if we are to have social and political stability in the next generation. Nobody should be fooled about the prospects for social growth in societies where 50 percent of the people under 30 are unemployed. This is not going to be resolved by giving them assembly line car-building jobs. Everybody sees that. Governments are collectively throwing up their hands about what to do about the situation. Hence, the rapidity with which, in systems of proportional representation, young people are giving up on established political parties. When the Pirates can take 8.3% of the vote in Schleswig-Holstein, it is already clear that young people realize that established political policy-making is not going to be directed at their future economic welfare. And we need to listen, democratically, to the large number of young people around the world who insist that internet freedom and an end to snooping and control is necessary to their welfare and ability to create and live.

Disintermediation means there will be more service providers throughout the economy with whom we are directly in touch. That means more jobs outside hierarchies and fewer jobs inside hierarchies. Young people around the world whether they are my law students about to get a law license, or computer engineers about to begin their practices, or artists, or musicians, or photographers, need more freedom in the net, and more tools with which to create innovative service delivery platforms for themselves. A challenge to which their elders would not have risen successfully in 1955, but we are new generation of human beings working under new circumstances, and those rules have changed. They know the rules have changed. The indignados in every square in Spain know the rules have changed. It’s their governments that don’t know.

Which brings us I will admit to back to this question of anonymity, or rather, personal autonomy. One of the really problematic elements in teaching young people, at least the young people I teach, about privacy, is that we use the word privacy to mean several quite distinct things. Privacy means secrecy, sometimes. That is to say, the content of a message is obscured to all but it’s maker and intended recipient. Privacy means anonymity, sometimes, that means messages are not obscured, but the points generating and receiving those messages are obscured. And there is a third aspect of privacy which in my classroom I call autonomy. It is the opportunity to live a life in which the decisions that you make are unaffected by others’ access to secret or anonymous communication.

There is a reason that cities have always been engines of economic growth. It isn’t because bankers live there. Bankers live there because cities are engines of economic growth. The reason cities have been engines of economic growth since Sumer, is that young people move to them, to make new ways of being. Taking advantage of the fact that the city is where you escape the surveillance of the village, and the social control of the farm. “How you gonna keep them down on the farm after they’ve seen Paris?” was a fair question in 1919 and it had a lot do with the way the 20th century worked in the United States. The city is the historical system for the production of anonymity and the ability to experiment autonomously in ways of living. We are closing it.

Some years ago, to wit, at the beginning of 1995 we were having a debate at the Harvard Law School about Public Key Encryption. Two on two. On one side Jamie Gorelick, then the Deputy Attorney General of the United States, and Stewart Baker, then as now at Steptoe & Johnson when he isn’t in the United States government making horrendous social policy. On the other side, Danny Weitzner, now in the White House, and me. And we spent the afternoon talking back and forth about whether we should have to escrow our keys with the United States government, whether the clipper chip was going to work and many other very interesting subjects now as obsolete as Babylonia. And after it was all over, we walked across the Harvard campus for dinner at the Harvard Faculty Club and on the way across the campus Jamie Gorelick said to me “Eben, on the basis of nothing more than your public statements this afternoon I have enough to order the interception of your telephone conversations.” In 1995 that was a joke. It was a joke in bad taste when told to a citizen by an official of the United States Justice Department. But it was a joke. And we all laughed because everybody knew you couldn’t do that.

So we ate our dinner, and the table got cleared and all the plates went away, and the port and walnuts got scattered around, and Stewart Baker looked up and said “alright, we’ll let our hair down”, and he had none then and he has none now, but “we’ll let our hair down” Stewart said, “we’re not going to prosecute your client Mr Zimmerman. We’ve spent decades in a holding action against Public Key Encryption it’s worked pretty well but it’s almost over now, we’re gonna let it happen.” And then he looked around the table and he said, “but nobody here cares about anonymity do they?” A cold chill went up my spine.

And I thought, “OK, Stewart, I understand how it is. You’re going to let there be Public Key Encryption because the bankers are going to need it. And you’re going to spend the next 20 years trying to stop people from being anonymous ever again, and I’m going to spend those 20 years trying to stop you.” So far I must say from my friend Mr. Baker has been doing better than I had hoped, and I have been doing even worse than I had feared. Partly because of the thug in a hoodie, and partly for other reasons. We are on the verge of the elimination of the human right to be alone. We are on the verge of the elimination of the human right to do your own thinking, in your own place, in your own way without anybody knowing. Somebody in this room just proved a couple of minutes ago that if he shops at a particular web-store using one browser, he gets a different price than on the other. Because one of the browsers is linked to his browsing history. Prices, offers, commodities, opportunities, are now being based upon the data mining of everything. A senior government official in this government said to me after the United States changed its rules about how long they keep information on everybody about whom nothing is suspected – you all do know about that right? Rainy Wednesday on the 21st of March, long after the close of business, Department of Justice and the DNI, that’s the Director of National Intelligence, put out a joint press release announcing minor changes in the Ashcroft rules, including a minor change that says that all personally identifiable information in government databases at the National Center for Counter-Terrorism that are based around people of whom nothing is suspected, will no longer be retained as under the Ashcroft rules for a maximum of 180 days, the maximum has now been changed to 5 years. Which is infinity.

I told my students in my classroom, the only reason they said 5 years was they couldn’t get the sideways eight into the font for the press release, so they used an approximation. So I was talking to a senior government official of this government about that outcome and he said well you know we’ve come to realize that we need a robust social graph of the United States. That’s how we’re going to connect new information to old information. I said let’s just talk about the constitutional implications of this for a moment. You’re talking about taking us from the society we have always known, which we quaintly refer to as a free society, to a society in which the United States government keeps a list of everybody every American knows. So if you’re going to take us from what we used to call a free society to a society in which the US government keeps a list of everybody every American knows, what should be the constitutional procedure for doing this? Should we have, for example, a law? He just laughed. Because of course they didn’t need a law. They did it with a press release on a rainy Wednesday night after everybody went home, and you live there now.

Whether it is possible to have innovation under conditions of complete despotism is an interesting question. Right-wing Americans or maybe even center-right Americans, have long insisted that one of the problems with 20th century totalitarianism, from which they legitimately distinguish themselves, was that it eliminated the possibility of what they call free markets and innovation. We’re about to test whether they were right.

The network, as it stands now, is an extraordinary platform for enhanced social control. Very rapidly, and with no apparent remorse, the two largest governments on earth, that of the United States of America and the People’s Republic of China have adopted essentially identical points of view. A robust social graph connecting government to everybody and the exhaustive data mining of society is both governments fundamental policy with respect to their different forms of what they both refer to, or think of, as stability maintenance. It is true of course that they have different theories of how to maintain stability for whom and why, but the technology of stability maintenance is becoming essentially identical.

We need, we, who understand what is happening, need, to be very vocal about that. But it isn’t just our civil liberties that are at stake, I shouldn’t need to say that, that should be enough, but of course it isn’t. We need to make clear that the other part of what that costs us is the very vitality and vibrancy of invention culture and discourse, that wide open robust and uninhibited public debate that the Supreme Court so loved in New York Times against Sullivan. And that freedom to tinker, to invent, to be different, to be non-conformist for which people have always moved to the cities that gave them anonymity, and a chance to experiment with who they are, and why they can do.

This more than anything else, is what sustains social vitality and economic growth in the 21st century. Of course we need anonymity for other reasons. Of course we are persuing something that might be appropriately described as protection for the integrity of the human soul. But that’s not government’s concern. It is precisely the glory of the way we understand civil society that that is not government’s concern. It is precisely our commitment to the idea of the individual’s development at her own pace, and in her own way, that has been the centerpiece of what we understood to be our society’s fundamental commitment that means that the protection for the integrity of the human soul is our business, not the government’s business. But government must attend to the material welfare of its citizens and it must attend to the long run good of the society they manage. And we must be clear to government that there is no tension between the maintenance of civil liberty in the form of the right to be let alone, there is no distinction between the civil liberty policy of assuring the right to be let alone, and the economic policy of securing innovation under austerity. They require the same thing.

We need Free Software, we need Free Hardware we can hack on, we need Free Spectrum we can use to communicate with one another, without let or hindrance. We need to be able to educate and provide access to educational material to everyone on earth without regard to the ability to pay. We need to provide a pathway to an independent economic and intellectual life, for every young person. The technology we need, we have, I have spent some time and many people in this room, including Isaac have spent more time now, trying to make use of cheap, power efficient compact server computers, the size of AC chargers for mobile phones, which with the right software we can use to populate the net with robots that respect privacy. Instead of the robots that disrespect privacy which we now carry in almost every pocket.

We need to retrofit the first law of robotics into this society within the next few minutes or we’re cooked. We can do that. That’s civil innovation. We can help to continue the long lifetime of general purpose computers everybody can hack on. By using them, by needing them, by spreading them around. We can use our own force as consumers and technologists to deprecate closed networks and locked-down objects, but without clear guidance in public policy we will remain a tiny minority, 8.3% let’s say. Which will not be sufficient to lift us out of the slough into which the bankers have driven us.

Innovation under austerity is our battle-cry. Not a battle-cry for the things we most care about, but the ones the other people most care about. Our entree to social policy for the next five years, and our last chance to do in government what we have not been able to do by attempting to preserve our mere liberties. Which have been shamefully abused by our friends in government as well as by our adversaries. We have been taken to the cleaners with respect to our rights, and we have been taken to the cleaners with respect to everybody’s money.

I wish that I could say that the easiest thing to do was going to be to get our freedoms back, it isn’t. Nobody will run in the election this year on the basis of the restoration of our civil liberties. But they will all talk about austerity and growth. And we must bring our message where they are.

That’s my first draft. Inadequate in every way, but at least a place to start. And if we have no place to start, we will lose. And our loss will be long. And the night will be very dark.

Thank you very much.

Thank you that’s very kind of you, now let’s talk about it.

Building a Semantic Graph of FrameNet 1.6

FrameNet 1.6

In October 2015, we requested and obtained a research license for the FrameNet 1.6 database from the International Computer Science Institute at Berkeley.

The FrameNet 1.6 distribution includes the following files and folders:

Files/Folders  Contents
frameIndex.xml, frameIndex.xsl Index of Frames in /frames (below) plus corresponding stylesheet.
frRelation.xml Index of Frame Relations.
luIndex.xml, luIndex.xsl Index of Lexical Units in /lu (below) plus corresponding stylesheet
fulltextIndex.xml, fulltext.xsl Index of fully Annotated Texts in /fulltext (below) plus corresponding stylesheet
semTypes.xml XML file of Semantic Types and their definitions.
/frame XML files for 1,206 (or 1,205) Frames – filenames in the form <Frame name>.xml, where <Frame name> varies from “Abandonment” to “Working_a_post”.
/lu XML files for 13,189 Lexical Units – filenames in the form lu<N>.xml, where <N> varies from “2” to “18184”
/schema XML Schema Definition (XSD) – some discussed at length below.
/fulltext XML files for 100 fully annotated texts.
/miscXML lemma_to_wordform.xml and lexeme_to_wordform.xml – two supplementary XML files containing all the mappings of word forms to lexemes and lemmas in FrameNet 1.6; and DifferencesR1.5-R16.xml – an XML file highlighting the differences between FrameNet 1.6 and 1.5.
/docs * User manual for FrameNet
* General release notes for FrameNet 1.6
* XML release notes for FrameNet 1.5

FrameNet Explorer 3

The FrameNet Explorer (FNE) was developed as part of the CL Research Dictionary Maintenance Program (DIMAP) version 3. The FNE builds the DIMAP Dictionary using the following FrameNet 1.6 resources:

  • frameIndex.xml
  • frRelation.xml
  • XML files for Frames in the /frame directory
  • XML files for Lexical Unit in the /lu directory.

 

Frames

Selecting the Tab:Frame in the FNE displays columns for:

  • Name of the Frame
  • Numeric ID of the Frame
  • Number of Frame Elements in the Frame
  • Number of Lexical Units that have been used with the Frame, and
  • Number of Annotated Sentences for the Frame.

Selecting a particular Frame by Name displays additional information about the Frame:

  • Description or “Definition” of the Frame
  • Names of the Frame Elements in the Frame
  • Lexemes in the Frame, and
  • Frame-to-Frame Relations that the Frame participates in.

Frame Elements

Selecting the Tab:Frame in the FNE displays columns for:

  • Name of the Frame Element
  • Number of Frames in which identically-named Frame Elements have been used (the same Name does not imply the same meaning for the Frame Element)

Selecting a particular Frame Element by Name displays additional information about the Frame Element:

  • The Frame in which it appears
  • Type of Frame Element (Core, Peripheral, or extra-thematic), and
  • Description or “Definition” of the Frame Element for that Frame.

Lexical Units

Selecting the Tab:Lexical Units in the FNE displays columns for:

  • Name of the Lexical Unit
  • Numeric ID of the Lexical Unit (essentially the file name for the lexeme)
  • Part of Speech of the Lexical Unit
  • Number of Annotated Sentences for the Lexical Unit
  • Name of the Frame the Lexical Unit appears in
  • Numeric ID of the Frame the Lexical Unit appears in
  • Description or “Definition” of the Lexical Unit – sourced either to FrameNet (FN) or to the Concise Oxford Dictionary (COD).

Selecting a particular Lexical Unit by Name displays additional information about the Lexical Unit:

  • Annotated Sentences for the Lexical Unit, and
  • Numeric IDs for the Annotated Sentences.

Exporting Dictionaries and Definitions

Once we have imported the the FrameNet 1.6 database into the FrameNet Explorer, we can export three files:

  • frames.dmp – a dictionary of Frames via Tab:Sample Selection
  • FEdefs.csv – definitions of Frame Elements via Tab: Frame Elements
  • fnlex.dmp – a dictionary of Lexical Units via Tab: Sample Selection

Processing Dictionaries and Definitions

Frames Dictionary

<to be updated>

Frame Elements Definitions

<to be updated>

Lexical Units Dictionary

<to be updated>

Complexity and Governance – 2013 Conference

Nanyang Technological University
July 15 – 19, 2013

Complexity Lens – 2015 Conference

Nanyang Technological University
July 9 – 10, 2015

Emerging Patterns – 2015 Conference

Nanyang Technological University
March 2 – 4, 2015

Robert Axelrod

Publications

Presentations

Accessing services

Possible pathway to service

  1. Apply to CSSS, who refers to ->
  2. Family doctor, who refers to ->
  3. CSSS, who refers to ->
  4. Evaluation-Liaison Module (Douglas), who refers to ->
  5. Anxiety disorders clinic (Douglas), who refers back to ->
  6. CSSS/Family doctor

Accessing services at the Douglas Mental Health University Institute

  1. Accessing a family doctor through CSSS – Main website (French)
  2. Referrals to Douglas Mental Health University Institute
  3. Anxiety disorders clinic at Douglas Institute

Accessing services at the Royal Victoria Hospital

  1. Department of Psychiatry
  2. McGill Department of Psychology at the Royal Victoria Hospital

OECD – Median Disposable Income – Working Age Population 18 – 65 years – 2012

Demographics

The OECD defines the “Working Age” as 18 – 65 years old. Table 1 of OECD – Labour Force Statistics provides the total population for 31 countries of interest in 2012 . We use these figures and the Age group share data in OECD – Age group share, 18-25 year oldsOECD – Age group share, 26-40 year oldsOECD – Age group share, 41-50 year olds, and OECD – Age group share, 51-65 year olds to derive numbers of individuals within four exhaustive, mutually exclusive categories of Working Age individuals:

Country 18-25
000s
26-40
000s
41-50
000s
51-65
000s
Total
000s
Australia 2,545 4,908 3,181 4,067 14,702
Austria 876 1,516 1,407 1,702 5,502
Belgium 1,068 2,192 1,624 2,169 7,055
Canada 3,801 7,150 5,266 7,115 23,334
Czech Republic 1,008 2,448 1,408 2,154 7,020
Denmark 570 1,028 816 1,068 3,483
Estonia 153 284 176 251 866
Finland 514 1,007 714 1,142 3,378
France 5,526 12,132 8,956 12,513 39,127
Germany 6,880 14,826 12,942 17,529 52,178
Greece 887 2,328 1,630 2,107 6,953
Hungary 1,170 2,073 1,150 2,172 6,567
Iceland 36 63 42 58 200
Ireland 444 976 582 839 2,842
Israel 917 1,685 846 1,162 4,612
Italy 4,901 12,042 9,682 11,618 38,245
Luxembourg 51 113 85 92 343
Mexico 16,504 24,932 13,461 14,514 69,413
Netherlands 1,641 3,082 2,597 3,384 10,706
New Zealand 505 851 642 802 2,801
Norway 537 1,018 727 913 3,197
Poland 4,123 9,016 4,855 8,246 26,241
Portugal 904 2,187 1,566 2,050 6,708
Slovak Republic 708 1,189 767 1,092 3,758
Slovenia 197 442 310 423 1,373
Spain 3,788 10,802 7,669 8,464 30,725
Sweden 1,009 1,761 1,313 1,618 5,701
Switzerland 815 1,599 1,311 1,567 5,294
Turkey 9,021 18,568 9,697 9,923 47,210
United Kingdom 7,071 12,613 9,364 11,530 40,580
United States 34,526 61,519 42,686 60,263 198,996

Table 1. Census within four Age groups of Working Age (18-65 years old) individuals. Source: OECD.

Disposable Income

The OECD provides Median Disposable Income data (using the new definition of income) for Working Age Population 18 – 65 years old in 30 of 35 countries surveyed in 2012. Our analysis also includes the corresponding figure for Canada in 2011. 1

Table 2 presents the Median Disposable Income for the Working Age Population in these 31 countries – expressed in the national currency and in US dollars:

Country National
Currency
Median
Disposable Income (MDI)
National Currency per US dollar MDI
(US dollars)
Australia Australian Dollar 51,242 1.52809 $33,533
Austria Euro 26,481 0.85858 $30,843
Belgium Euro 26,084 0.89261 $29,222
Canada‡ Canadian Dollar 38,248 1.28458 $29,775
Czech Republic Czech Koruna 235,111 14.5582 $16,150
Denmark Danish Krone 246,775 8.40991 $29,343
Estonia Euro 8,149 0.6211 $13,120
Finland Euro 27,154 0.98624 $27,533
France Euro 22,318 0.88583 $25,194
Germany Euro 22,480 0.81703 $27,514
Greece Euro 9,629 0.75303 $12,787
Hungary Forint 1,592,591 143.793 $11,076
Iceland Iceland Krona 4,145,840 142.563 $29,081
Ireland Euro 22,495 0.96521 $23,306
Israel New Israeli Sheqel 78,690 4.25607 $18,489
Italy Euro 18,572 0.83991 $22,112
Luxembourg Euro 38,559 0.96569 $39,929
Mexico Mexican Peso 50,269 9.17818 $5,477
Netherlands Euro 23,200 0.88103 $26,333
New Zealand New Zealand Dollar 41,881 1.57817 $26,538
Norway Norwegian Krone 361,039 9.72688 $37,118
Poland Zloty 26,435 1.91796 $13,783
Portugal Euro 9,981 0.66846 $14,931
Slovak Republic Euro 8,549 0.5719 $14,948
Slovenia Euro 14,052 0.67771 $20,735
Spain Euro 15,908 0.77144 $20,621
Sweden Swedish Krona 266,301 9.1096 $29,233
Switzerland Swiss Franc 59,621 1.55332 $38,383
Turkey Turkish Lira 12,000 1.25179 $9,586
United Kingdom Pound Sterling 17,648 0.76649 $23,024
United States US Dollar 32,819 1.0000 $32,819

Table 2. Median Disposable Income, Working Age Population 18 – 65 years old in 2012. ‡ Canadian figures are for 2011. Source: OECD. 

Disposable Income Ratios

We use the P50/P10 ratio and the P90/P50 ratio to derive the upper bound value of the first and ninth decile, respectively, from the Median (P50) Disposable Income (US dollars) in Table 2:

Country P50/P10 P90/P50 P10 P50 P90
Australia 2.3 1.9 $14,580 $33,533 $63,713
Austria 2.0 1.7 $15,421 $30,843 $52,432
Belgium 2.1 1.6 $13,915 $29,222 $46,755
Canada‡ 2.3 1.9 $12,945 $29,775 $56,572
Czech Republic 1.7 1.8 $9,500 $16,150 $29,069
Denmark 1.8 1.6 $16,302 $29,343 $46,949
Estonia 2.4 2.1 $5,467 $13,120 $27,553
Finland 1.9 1.7 $14,491 $27,533 $46,806
France 1.9 1.9 $13,260 $25,194 $47,869
Germany 2.0 1.8 $13,757 $27,514 $49,526
Greece 2.7 1.9 $4,736 $12,787 $24,295
Hungary 2.1 1.8 $5,274 $11,076 $19,936
Iceland 1.8 1.7 $16,156 $29,081 $49,437
Ireland 2.0 2.0 $11,653 $23,306 $46,612
Israel 2.7 2.1 $6,848 $18,489 $38,827
Italy 2.4 1.9 $9,213 $22,112 $42,013
Luxembourg 1.9 1.9 $21,015 $39,929 $75,865
Mexico 2.9 2.8 $1,889 $5,477 $15,336
Netherlands 2.0 1.8 $13,166 $26,333 $47,399
New Zealand 2.2 2.0 $12,063 $26,538 $53,076
Norway 2.1 1.6 $17,675 $37,118 $59,388
Poland 2.1 1.9 $6,563 $13,783 $26,187
Portugal 2.4 2.1 $6,221 $14,931 $31,356
Slovak Republic 1.9 1.7 $7,868 $14,948 $25,412
Slovenia 1.9 1.6 $10,913 $20,735 $33,175
Spain 2.5 2.0 $8,249 $20,621 $41,243
Sweden 2.1 1.7 $13,920 $29,233 $49,696
Switzerland 1.9 1.7 $20,202 $38,383 $65,251
Turkey 2.4 2.4 $3,994 $9,586 $23,007
United Kingdom 2.2 2.0 $10,466 $23,024 $46,049
United States 2.9 2.2 $11,317 $32,819 $72,202

Table 3. P50/P10  and P90/P50 ratios; P10, P50 (Median), and P90 values of  Disposable Income (US dollars), Working Age Population (18 – 65 year olds) in 2012. ‡ Canadian figures are for 2011. Source: OECD.

OECD Labour Statistics provides two views of total employment:

  • Table 20: Total Employment, including Armed Forces – too many missing entries
  • Table 22: Total Civilian Employment

Using Total Civilian Employment figures, we derive the number of civilian employees whose Disposable Income equals or exceeds the national P90 value  in 2012:

>

Country P90 Civilian Earners >=P90 000s
Australia $63,713 1,134
Austria $52,432 417
Belgium $46,755 452
Canada‡ $56,572 1,750
Czech Republic $29,069 487
Denmark $46,949 268
Estonia $27,553 61
Finland $46,806 247
France $47,869 2,623
Germany $49,526 3,989
Greece $24,295 379
Hungary $19,936 386
Iceland $49,437 16
Ireland $46,612 182
Israel $38,827 335
Italy $42,013 2,263
Luxembourg $75,865 37
Mexico $15,336 4,900
Netherlands $47,399 842
New Zealand $53,076 221
Norway $59,388 258
Poland $26,187 1,559
Portugal $31,356 460
Slovak Republic $25,412 232
Slovenia $33,175 92
Spain $41,243 1,753
Sweden $49,696 465
Switzerland $65,251 477
Turkey $23,007 2,482
United Kingdom $46,049 2,926
United States $72,202 14,246

Table 4. Civilian employees whose Disposable Income (US dollars) equals or exceeds the national P90 value in 2012. ‡ Canadian P90 figure is for 2011. Source: OECD.

  1. We exclude Chile, Japan, Korea, and Russia from our analysis, as there were no figures available based on the new income definition.

IBM Healthcare Could Have Done Better Today

Today @IBMHealthcare tweeted this …

‏@IBMHealthcare Beyond the basics: Crafting an in-depth #healthcare #security strategy

… which linked to IBM’s Security Thought Leadership White Paper Healthcare Securing the healthcare enterprise: Taking action to strengthen cybersecurity in the healthcare industry (March 2015).

While I can’t comment on IBM’s business solutions “to strengthen cybersecurity in the healthcare industry,” I am surprised at the quality of information that IBM relies on to describe “the nature of today’s cyber attackers” to its potential customers.

For example, IBM presents a figure (reproduced below) and references a CNN Money report, Hospital network hacked, 4.5 million records stolen (August 18, 2014).

Leading source of data leaks in healthcare institutions
Figure 1. IBM’s leading source of data leaks in healthcare institutions

In fact, CNN is not the source for Figure 1. Another IBM publication, MSS Industry overview – Healthcare: Research and intelligence report (October 7, 2014) presents the same figure, and references “Chronology of Data Breaches Security Breaches 2005-Present, Privacy Rights Clearinghouse.” IBM seems to have generated Figure 1 by querying an API on the Privacy Rights Clearinghouse website.

I wonder why IBM does not use authoritative, readily available data on breaches of protected health information to make its business case and to educate the public.

For instance, a research letter (Liu, Musen & Chou, 2015) published recently in the Journal of the American Medical Association1 described breaches of protected health information that had been reported from 2010 through 2013 by entities covered by the Health Insurance Portability and Accountability Act in the United States . Under the Health Information Technology for Economic and Clinical Health Act (2009), breaches involving the acquisition, access, use, or disclosure of protected health information and thus posing a significant risk to affected individuals must be reported.

Recently, we extended the original dataset of Liu et. al. to include breaches of health information up to the present. Table 1 summarizes the number of incidents and victims of breaches of health information in the United States from January 2010 to August 2015, inclusive.

Counts and Victims of Health Information Breaches - US 2010-2015
Table 1. Number of incidents and victims of breaches of health information. † 2015 data are for January – August inclusive only.

Notice the tremendous spike in the number of victims in 2015 – a dramatic development that IBM took no note of today.

Figure 2 depicts the distribution of victims/breach of health information as a series of boxplots.

Distribution of number of victims/incident (log scale) of breach of health information U.S. 2010-2015
Figure 2. Distribution of victims/incident (log scale) of breach of health information. † 2015 data are for January – August inclusive only.

We see that in seventy-five percent of all incidents, the number of victims/breach over the year has fallen consistently below 104 (10,000). A small number of incidents have involved 100,000 – 1,000,000 victims/breach, and an even smaller number have involved 1,000,000 – 10,000,000 victims/breach. Incidents involving more than 10,000,000 victims/breach made their first appearance in 2015.

 

In light of these dramatic developments, it’s a shame that IBM is relying on outdated information when it comes to educating the public and identifying potential solutions “to strengthen cybersecurity in the healthcare industry.”

 

  1.  Liu V, Musen MA, Chou T. Data Breaches of Protected Health Information in the United States. JAMA. 2015;313(14):1471-1473. doi:10.1001/jama.2015.2252.

Cyber Insurance – Readings

Cambridge Centre for Risk Studies

Miscellaneous

News

Toronto Citizen’s Arrest of South Korea’s Smart Sheriff

From The Citizen Lab, Are the Kids Alright? Digital Risks to Minors from South Korea’s Smart Sheriff Application, Appendix B: Legal and Policy Issues (2015)

South Korea is one of the most highly connected countries in the world when it comes to Internet and mobile phone access. Whereas 36.2 percent of Korean minors had smartphones in 2011, the number grew to 81.5 percent within two years, with high penetration rates even among elementary school children.

The South Korean government has taken steps to regulate the use of digital media among minors, maintaining a “shutdown” rule that restricts access to online gaming for minors under the age of sixteen after midnight.

In 2013, regulators began focusing on combating excessive smartphone use, requiring that schools organize “boot camps” where no Internet usage is allowed, teach classes on Internet addiction, and educate those as young as three on how to prevent overuse of digital devices and the Internet.

By 2014, schools were piloting a program that required students, with parental approval, to download an application that allowed teachers to remotely track and control students’ smartphones, including the ability to lock the phone or allow only emergency calls.

By April 2015, the Korean government enacted a new measure requiring telecommunications business operators that enter into service contracts with minors to provide a means of blocking harmful content on the minor’s mobile device and ensure that parents receive notifications whenever the blocking means becomes inoperative. This measure has ushered in the wide-ranging use of parental monitoring software, with Smart Sheriff one of the most prominent options for fulfilling the mandate. One month into the mandate, these applications were reportedly downloaded at least 480,000 times.

With cooperation on implementation from numerous entities in the public and private sector, the new requirements constitute a pervasive parental monitoring and control mandate.

While Smart Sheriff is not the only tool offered to support compliance with the new regulations on provision of means to block harmful content, the Korean government appears to have uniquely supported its development and promotion.

According to its terms of use, Smart Sheriff collects and retains for one year information about applications installed on the child’s smartphone, data related to account password, member name, phone number, child’s date of birth, IP addresses of service access, and log file information such as access time.

Smart Sheriff’s terms of use also provide for sharing the student’s data with the Office of Education and the student’s school for purposes of smartphone addiction counselling, and with telecommunications business operators for the purpose of complying with the notification obligations of the mandate on installation of means for blocking harmful content.

What could go wrong?

 

Wages – OECD data and analyses

A young fellow I know was looking to model a sample of conspicuous consumers’ online behavior. We got to speaking about the prosperity of market segments across the globe, and that led me to consider parts of the Online OECD Employment database.

Definition

Average annual wages are obtained by dividing the national-accounts-based total wage bill by the average number of employees in the total economy, which is then multiplied by the ratio of the average usual weekly hours per full-time employee to the average usually weekly hours for all employees. This indicator is measured in USD constant prices using 2012 base year and Purchasing Power Parities (PPPs) for private consumption of the same year. Read more details on the estimation of average annual wages.

Links:

http://www.oecd-ilibrary.org/employment/average-wages/indicator/english_cc3e1387-en

Definitions

Earnings dispersion: this dataset contains three earnings-dispersion measures – ratio of 9th-to-1st, 9th-to-5th and 5th-to-1st – where ninth, fifth (or  median) and first deciles are upper-earnings decile limits, unless otherwise indicated, of gross earnings of full-time dependent employees. The incidence of low pay refers to the share of workers earning less than two-thirds of median earnings. The incidence of high pay refers to the share of workers earning more than one-and-a-half time median earnings.

Gaps: The gender wage gap is calculated as the difference between median earnings of men and women relative to median earnings of men. The age wage gap is calculated as the difference between mean earnings of 25-54 year-olds and that of 15-24 year-olds (respectively 55-64 year-olds) relative to mean earnings of 25-54 year-olds.  Earnings by skill (or education levels) refer to mean annual earnings of full-time full-year 25-64 year-old employees. Earnings gaps by skill levels are calculated as the difference between mean earnings of medium-skilled employees and low- (respectively high-) skilled employees relative to mean earnings of medium-skilled employees.

Links:

In a later post, we will look at deriving indices from the run-of-the-mill OECD datasets that might characterize the sort of prosperity that marks the conspicuous consumer.

Also see:

What’s the sense of Service-Dominant Logic?

My interest in specifying a model of public service has involved me in the study of more general notion of service. I have been most taken with two models of service, in particular:

  • Resource-Event-Agent model (REA)
  • Service-Dominant Logic (S-DL)

The scholarship associated with the Resource-Event-Agent model has included a substantial investment in specifying an REA ontology using both a formal language representation (the Web Ontology Language or OWL) and a graphical representation (the Unified Modeling Language or UML Profile for OWL). The International Standards Organization has incorporated REA into the Open-edi Business Transaction Ontology (OeBTO).

The scholarship associated with Service-Dominant Logic has been less concerned with formal language representation, and more concerned with bringing a certain perspective to bear on the widest-possible range of human behavior – aspiring to be a “unifying and transcending view of business and, more broadly, economic and social organization” (Lusch and Vargo, 2014). The status of Service-Dominant Logic as a model was enhanced when IBM proposed that Service-Dominant Logic should be the foundation of a new, multidisciplinary “science” of service  (Maglio and Spohrer 2008).

There is no doubt that Service-Dominant Logic has had a far greater impact on our thinking about service than has the Resource-Event-Agent model. The primacy of Service-Dominant Logic is even more evident when one considers the design, delivery, and innovation of service in the marketplace.

There is also an aspirational – indeed, an inspirational – quality to the scholarship associated with Service-Dominant Logic that is lacking in somewhat dry, technical approach of the scholarship associated with the Resource-Event-Agent model.

For all that, the formulation of Service-Dominant Logic needs to be tightened up – in the end, the rhetoric may be less inspiring, but it will also be less confusing.

Let’s first consider the formal elements of Service-Dominant Logic – taking Lusch and Vargo’s (2014) book-length treatment of their model and Vargo and Lusch’s (2015) update as our points of reference.

Institutional Logics

Lusch and Vargo begin with the proposition that “part of our nature as humans is to develop belief systems that become handy ways of seeing and understanding the world around us and for ordering our reality.” [Ch 1] Belief systems enable viewing a complex world in what promises to be coherent terms and provide a lens for perceptually separating noise from signal. Thus, belief systems contribute to comfort, understanding and sense-making.

Belief systems become normative and play a key role in guiding and determining our behavior. Lusch and Vargo promote at least some belief systems to the status of “institutional logics”.1 In 2004, they began to formulate a Service-Dominant Logic (S-D Logic) “to contribute to the understanding of the world of economic and social exchange among human actors.” Service-Dominant Logic is a “mindset” that offers an alternative “worldview” to traditional Goods-Dominant Logic (G-D Logic).

Lusch and Vargo have associated Service-Dominant Logic with a set of terms they call a Lexicona set of propositions they call Foundational Premises, and a set of Axioms that represent a subset of the Foundational Premises.

In 2014, Lusch and Vargo identified ten Foundational Premises including four Axioms. In 2015, Vargo and Lusch introduced a fifth Axiom (an eleventh FP) and revised the wording of one other Axiom and three other FPs. We will designate these two versions of the Axioms/Foundational Premises of Service-Dominant Logic as FP-2014 and FP-2015, respectively. The terms of Lexicon are arranged in a hierarchy; the Foundational Premises are also arranged in a hierarchy.

Lusch and Vargo (2014) present the hierarchical structure of the Lexicon of Service-Dominant Logic (hereafter referred to as Lexicon-2014) in Figure 1 …

Lusch and Vargo 2014 - Figure 3.2 S-D Logic lexicon
Figure 1 S-D Logic Lexicon-2014 (Lusch and Vargo 2014).

… and the hierarchical structure of FP-2014 in Figure 2.

Lusch and Vargo 2014 - Figure 3.1 Axioms and foundational premises of S-D Logic
Figure 2. Axioms and foundational premises of S-D Logic – aka FP-2014 – (Vargo and Lusch 2014).

The Lexicon is used to develop and elaborate the Axioms and Foundational Premises [57].

Let’s consider how meaningful or potentially confusing  this last point might be – and examine the overlap between the terms of the Lexicon-2014 and the terms used to express the FP-2014 and the FP-2015. Before we dive in to this exercise, we first need to identify how Lusch and Vargo use the word “lexicon” and then we need to bring in some basic concepts of semantics.

Lexicon

We would highlight six passages where Lusch and Vargo (2014) discuss their idea of the Lexicon of Service-Dominant Logic. Half of these passages speak of the general idea of Lexicon; the other half address the way a shift in thinking about a concept like Service motivates a shift in lexicon – and the difficulties that can and often do immediately ensue in efforts to communicate about the concept.

General idea of Lexicon

A key challenge we have faced in developing and communicating S-D logic is the precision of its lexicon. We soon realized how important words and language are in framing our view and conceptualization of the world and, hence, how it influences our actions or behavior. We found subtle yet important distinctions between terms such as "services" versus "service," "customers" versus "consum­ers," static and tangible resources and dynamic and intangible resources. Thus, much of what is to be learned from this book concerns how to uncompact new and/or revised meanings for old terms - for example, what are a resource, cocreation, and value? But we also found it necessary to develop new "concepts" and language, which we will introduce and explain here. These include "service ecosystems," "resource integration," "resourceness," and "value-in-context." We believe that, although it will take some effort to develop an understanding of the lexicon, most readers will find it worthwhile. [xxii]
All logics are based on premises and assumptions. Often these are not explicit or spoken but are implicit and unspoken. Logics can be observed in everyday practices and language. In the development of service-dominant (S-D) logic we have attempted to be explicit about its premises, assumptions, and language (or what we call its lexicon). [53]
Theories and models are abstractions of reality. Language and words are used to develop abstractions, and these abstractions are then related to each other in order to describe or explain the phenomena of interest. The goal is to be parsimonious, while still being as isomorphic as possible. This implies using as few concepts as necessary to describe and explain the phenomena of interest; while, at the same time, striving for correspondence between the theory or model and real-world phenomena. Predictably, it is very difficult to be both parsimonious and isomorphic with the same theory, model, or logic. We suggest that S-D logic strikes a reasonably good balance between these two objectives.

As noted, we believe S-D logic is reasonably parsimonious in its lexicon. In fact, it deals with only four core, foundational concepts (actors, service, resour­ces, value), and from these we derive ten additional concepts. [55 - also footnote 2 gives "The lexicon of S-D logic is continuing to develop and includes many other emerging terms such as service ecosystems ..."]

A shift in thinking that motivates a shift in Lexicon

All logics also have a lexicon developed by the community that supports and uses the logic. The lexicon comprises the terms and concepts, represented through words or symbols, which communicate meaning and help to coor­dinate thought among the community. To understand S-D logic, its axioms, and FPs, it is critical to become familiar with its lexicon. This can be difficult because some of the language is similar to the language of G-D logic, albeit with nuanced meanings. [54]
From the rapid ascendance and impact of the services marketing and management literature in the 1970s and 1980s, we began to see other currents of change in thinking. As is often the case, when thinking starts to change, it is supplemented or leveraged by the emergence of a new lexicon, which, in turn, further influences thinking and ultimately behavior or action. [202]
Part of the difficulty of mastering S-D logic is the enduring, strong pull of G-D logic. G-D logic is not only embedded in many organizational routines and practices, it is also embodied in our minds, and practices and institutions of society. In fact, even after nearly two decades of intense work on S-D logic and its associated lexicon, we still find ourselves occasionally slipping back to the G-D logic mindset and lexicon. Be forewarned: it takes work and training to see every firm offering, tangible or intangible, as just an input, something whose value is only realized in its use and in the context of and integration with resources from other sources. It is difficult to emancipate oneself from the restricted perspective of the firm-centric model, which treats customers as operand resources, whose role is to be captured for the net present value of their flow of financial resources to the enterprise - what, in G-D logic terms, is referred to as lifetime value of the customer, which is inappropriately paraded as "relationship" marketing. It is difficult not to think about the firm as the center of the wealth creation or as the producer and provider of value. It can be equally difficult to divorce oneself from that view that customers consume and destroy value. It is difficult not to think about innovation as something that primarily occurs in the laboratories and offices of the enterprise, as opposed to something that occurs throughout the service ecosystem, through the social and economic processes of resource inte­gration and service exchange. The pervasive influence of G-D logical lexicon and frameworks on all of the business and management disciplines is a hard one from which to break free. It is extremely difficult not to think about a profit shortfall as the fault of management and employees but rather as due to the inadequacy of the G-D logic model.

We believe commitment to S-D logic and its premises and lexicon, focusing on and understanding its nuances and fully grasping its transcending nature, will reveal not only new solutions to old problems but also unlimited and unbounded opportunities for market expansion and the creation of new markets. That is a fairly bold value proposition but one that we think is achievable through becom­ing untethered to G-D logic and mastering S-D logic. [204]

Terms and hierarchical structure of the S-DL Lexicon

Lusch and Vargo identify four “core, foundational concepts” – Actor, Service, Resource, and Value – in Lexicon-2014, and “derive” from these ten additional “concepts.” As we saw above, Lusch and Vargo characterize Service-Dominant Logic as “reasonably parsimonious” on the basis of the relatively few terms in its Lexicon [55]. It’s disconcerting that, in the same breath (or at least in a related footnote), Lusch and Vargo advise that “The lexicon of S-D logic is continuing to develop and includes many other terms such as service ecosystems …” It’s also disconcerting that Lusch and Vargo do not identify the place of a key term like “service ecosystem” in the hierarchical structure of the Lexicon (Figure 1).

Lexical semantics

In order to take a proper look at the terms of the S-DL Lexicon, we need to draw upon some model of language and meaning – and for this we want to set out some of the basic ideas of lexical semantics.

Wordform An orthographic or phonological form e.g. the written word “sing”, or the spoken word “songs”.
Sense The meaning associated with a wordform.
Lexeme A pairing of a wordform and (one of) its sense(s).
Lemma The grammatical form that is used to represent a lexeme. This is often the base form e.g. carpet is the lemma for “carpets”, or the infinitive form e.g. to sing is the lemma for “sang”.
Lemmatization The process of mapping a wordform to a lemma. Lemmatization is not always deterministic – it may depend on the context e.g. the wordform “found” can map to the lemma find (meaning “to locate”) or the lemma found (meaning “to create an institution”), and on part-of-speech e.g. the wordform “tables” has two possible lemmas, the noun “table” and the verb “table”. Each word sense is represented by placing a superscript on the orthographic form of the lemma, as in table1 and table2.
Lexicon A finite list of lexemes.

Table 1. Lexical semantics: some basic terms and definitions.

We will describe the procedures of lexical semantics only when we need to call upon them in our analysis of Lexicon-2014.

For now, let’s use some of the concepts of lexical semantics to recast Lusch and Vargo’s presentation of Lexicon-2014:2

Wordform Lemma FP-2014 FP-2015
Core concepts
“Actors” Actor 9 9
“Resources” Resource 4, 9 4, 9
“Service” Service 1, 3, 5, 8 1, 3, 5, 8
“Value” Value 6, 7, 10 6, 7, 10
Derived concepts
“Time-bound” Time
To bind1
“Relationally-bound” Relation 8 8
To bind2
“Resource-integrating” Resource 4, 9 4, 9
To integrate 9 9
“Operand” Operand
“Operant” Operant 4 4
“Goods” Good 3 3
“Currency” Currency
“Unique” Unique 10 10
“Co-created” Co- 6 6
To create 6 6
“Proposition” Proposition 7 7

Table 2. Mapping the “core concepts” and “derived concepts” of the Lexicon-2014 to their equivalent wordforms and lemmas.

A few notes about Table 2:

  • we allow that the sense of “to bind” as in “time bound” in different from the sense of “to bind” as in “relationally bound”
  • we associate a single lexeme – resource  with the two wordforms “resources” and “resource integrators”
  • we associate the wordform “cocreated” with two lexemes – to create and co- meaning “along with others

Finally, we note that four (or five) lexemes in Lexicon-2014  –  TimeOperand, Currency, To bind1 (and maybe To bind2) – are not put to use in expressing any members of FP-2014 or FP-2015.

Thus, some of the terms of Lexicon-2014 are not necessary for expressing the terms of FP-2014 or FP-2015.

Table 3 highlights those wordforms that are used to express the terms of FP-2014 and FP-2015 that have no apparent counterpart in Lexicon-2014. Clearly, the Lexicon-2014 is also not sufficient for expressing FP-2014, let alone FP-2015.

Axiom/FP Overlap with Lexicon-2014
FP1 – Axiom 1 2014/15: Service is the fundamental basis of exchange.
FP2 2014/15: Indirect exchange masks the fundamental basis of exchange.
FP3 2014/15: Goods are a distribution mechanism for service provision.
FP4 2014: Operant resources are the fundamental source of competitive advantage.
2015: Operant resources are the fundamental source of strategic benefit.
FP5 2014/15: All economies are service economies.
FP6 – Axiom 2 2014: The customer is always a co-creator of value.
2015: Value is co-created by multiple actors, always including the beneficiary.
FP7 2014: The enterprise can only make value propositions.
2015: Actors cannot deliver value but can participate in the creation and offering of value propositions.
FP8 2014: A service-centred view is customer oriented and relational.
2015: A service-centred view is inherently beneficiary oriented and relational.
FP9 – Axiom 3 2014/15: All economic and social actors are resource integrators.
FP10 – Axiom 4 2014/15: Value is always uniquely and phenomenologicaly determined by the beneficiary.
FP11 – Axiom 5 2015: Value co-creation is co-ordinated through actor-generated institutions and institutional arrangements.

Table 3. Wordforms of the FP-2014 and FP-2015 and their overlap with the Lexicon-2014 of Service-Dominant Logic. Non-overlapping wordforms in FP-2014 are highlighted in blue; additional non-overlapping wordforms in FP-2015 are highlighted in red.

Thus, we see clearly that the terms of the Lexicon are neither necessary nor sufficient for expressing the Foundational Premises of Service-Dominant Logic .

Next time we’ll take up the more complicated challenge of determining (if we can) the exact meaning that Lusch and Vargo want to associate with the lexemes of the Lexicon of Service-Dominant Logic.

References

Lusch, RF and Vargo, SL (2014), Service-Dominant logic: Premises, perspectives, possibilities, Cambridge University Press.

Vargo, SL and Lusch, RF (2015), Institutions and axioms: an extension and update of service-dominant logic, Journal of the Academy of Marketing Science, 1 – 19.

 

  1.   Friedland, R and Alford, RR – Bringing society back in: Symbols, practices, and institutional contradictions (1991).
  2. Lusch and Vargo are inclined to dispense with hyphens – we favour them, and have inserted them here into the wordforms “time-bound,” “relationally-bound,” “resource-integrators,” and “co-created.”

Communication and change in organizations (readings)

 

Communication and change in organizations

After Salem, P – The seven communication reasons organizations do not change (2008).

Organizational transformation involves changes in core features e.g. goals, authority relationships and organizational structure, markets, and technologies (Aldrich and Ruef, 2006; Rao and Singh, 1999). Management has made efforts to direct discontinuous second order change strategically (Nadler et al., 1995). These strategic initiatives have been successful only  about one third of the time (Cameron and Quinn, 1999; Meyer et al., 1995). Enduring improvement appears to be impossible without a change of culture (Cameron and Quinn, 1999, p. 9).

Culture is the set of embedded communication practices that distinguishes one group from another. Accomplishing transformational change involves replacing current competencies, routines, and rituals with other stable communication practices. Strategic initiatives whose purpose was to change the organization’s culture have succeeded less than 20 percent of the time (Smith, 2002). What this data suggest is that most legitimate systems – the established cultures – are robust and resistant to strategic initiatives. What management intends as transformational change may be integrated into the organization as simple adaptations.

The complexity of organizational change

The study of change in social systems has a long history. In the 1960s, Buckley (1967, 1968) argued that social systems continually experience natural tensions due to the variety in the system’s environment, to the variety and behaviors of the members within the system, and to the interaction between external and internal sources. The tension stimulates learning and the regrouping of components or actions. The changes may assist adaptation to various tensions, but they may also lead to goals, states, etc. the system has never experienced (Buckley, 1968). Buckley thought of society as a “complex adaptive system,” and he was concerned with how systems developed properties to insure their viability (Buckley, 1967, 1968, 1998). In 1968, Buckley hoped developments in mathematics would soon match the conceptual richness of these ideas.

Advances in non-linear dynamics would appear to be developments Buckley had desired. The two most recent bodies of work concern chaos theory and complexity theory (Holland, 1995; Kauffman, 1993, 1995). Both theories assume system interactions are part of an auto-catalytic process. Auto-catalytic or self-reinforcing processes have three properties:

  • the processes are iterative or repeated;
  • the processes are recursive, meaning the outputs for one iteration are the inputs for the next; and
  • the processes are multiplicative (i.e. non-linear or non-additive), suggesting that small effects may accumulate or aggregate to have bigger impacts later.

When researchers model processes as auto-catalytic, they employ formulas or algorithms with mathematical relationships that reflect these properties.

Weick’s sense-making model describes organizing as a function of such an auto-catalytic process. Sense is a function of a cue plus a frame plus a connection between the frame and the cue (Weick, 1995). However, the framing cycle does not occur once. It occurs repeatedly until individuals remove equivocality and make         plausible sense (Weick, 1979, 1995, 2001). When individuals communicate, they may make sense together, and so, communication draws attention to the social and cultural aspects of making sense. Sense-making involves a framing process that may reflect or may change culture. The frames may come from culture, and local sense-making may accumulate to alter cultural frames. When transformational change occurs, there are changes in cultural frames and communication practices.

The formulas also contain parameters or constants that determine the intensity of properties and, especially, the intensity of the interaction between properties or agents. Organizational researchers generally regard parameters as environmental conditions or as aspects of the strategic course of an organization (Thietart and Forgues, 1995). Common organizational parameters include:

  • leadership
  • the diversity of membership and organizational processes, and
  • the richness of the connectivity between social actors (Stacey, 1996).

These are common parameters, and a change in the critical values of these parameters would be necessary to reach a state where transformation was possible. These changes often accompany or are part of changes in core features mentioned in the introduction. Changes in second order parameters are inherent in major strategic initiatives and should produce transformational change. The initiatives, even the changes in second order parameters, have not produced the intended outcomes very often. Mostly, there was no change in the organization’s culture.

Chaos and complexity researchers refer to the result of one iteration of an auto-catalytic process as a phase and any pattern in a sequence of phases as an attractor. For example, a phase may be the configuration of agents after one iteration, and a repetition in a sequence of configurations would suggest an attractor. The pattern of phases around an attractor is a basin of attraction. Of course, the parameters, parameter values, and nature of the process itself limit what phases are possible. Chaos and complexity researchers refer to the range of all possible outcomes as a phase space. A particular account of a particular event would be comparable to a phase, and a pattern in several accounts would be an attractor (Stacey, 2001, 2003). The various accounts that lead to an organizing theme, the attractor, and the variations in the central theme would be part of a basin of attraction. A universe of discourse would be comparable to a phase space. One way of interpreting the failed efforts at transformational organizational change is to regard these strategic efforts as maintaining or just modifying the old organizing themes within the original universe of discourse.

A bifurcation point is a state of turbulence where second order change may be possible. The system may now move between at least one old basin of attraction and one new one (Polley, 1997). It is a time of great tension between the old and the new, and the system must “choose” its future (Prigogine and Stengers, 1984). Once at a bifurcation point, the system may move to one of five states:

  1. The old may dominate, and the system may return to the previous stable state
  2. The new may dominate, and the system may move to a new stable state
  3. The system may maintain a tension and oscillate between two or more states. This pattern may be a relatively stable pattern of oscillation between points, but it may involve so many points in a cycle that it may appear to be unstable.
  4. The system could pass through many bifurcation points, alternating patterns of stability and instability and leading to evolutionary changes in which one transformation builds on previous ones. A particular bifurcation point may be part of a transformational instability.
  5. The system might have passed through many bifurcation points leading to a continuous  unstable pattern. What appears to be random is limited or bounded by the auto-catalytic processes. The system’s “choice” at a particular bifurcation point depends on the general nature of the system, the history of past “choices,” parameters and their values, and the nature of auto-catalytic processes. Studying an organization at a bifurcation point would be an excellent way to learn about communication and organizational change.

Stacey (1996) described organizational change as conflict between a legitimate system and a shadow system. In this model, the natural tensions of everyday life drive an informal and emergent structure, the shadow system, and the accumulation of tensions may challenge the already dominant culture and formal structure, the legitimate system. Stacey’s description parallels Buckley’s (1967) older description of social change involving in- and out-groups. A change in parameter simply speeds the process and movement to one or a succession of bifurcation points.

The complexity of organizational change involves an accumulation of differences. Social actors construct novel behaviors or behaviors repeated with some modifications as part of auto-catalytic processes. Some auto-catalytic processes encourage greater novelty or modification while others discourage deviation. Various behaviors occur in relatively stable or unstable conditions. The tensions between these behaviors and the conditions are the basis for the relative stability of the social system, the system’s structure. Communication patterns may suggest underlying organizing themes, attractors, or there may be permutations around central themes, basins of attraction. The local activities of social actors may disrupt the tension and lead to a state, a bifurcation point, where the system may change its nature. That is, alternative basins of attraction may develop. The localized variety within the system, the shadow system, may naturally accumulate to challenge the established structure and process, the legitimate system. However, there may be some external disruption of parameters that stimulates the shadow system to challenge the legitimate system.

The communication reasons organizations do not change

Insufficient communication

When organizational members communicate during intense change, they will generate organizing themes about uncertainty or a lack of information about specific changes. Uncertainty is an inability to describe, predict, or explain (Salem and Williams, 1984), and complaints of inadequate information are common in organizations (Daniels and Spiker, 1983). However, information is not part of artifacts such as memos, reports, or websites. Organizational members create information and knowledge as they make sense (Salem, 2007; Weick, 1995). Communication is a social process in which individuals can make sense together, and artifacts are only an opportunity for making sense, an opportunity for conversation. Complaints about inadequate information are complaints about the lack of opportunities to make sense together.

Many approaches to change assume management will direct and control the process (Miller and Cardinal, 1994). Often, it is impossible to involve many people in making everyday decisions, and managers or a small group tend to simply “download” decisions to others. Management expects compliance, but this approach fails to gain acceptance or support for routine management decisions or decisions during change processes (Clampitt and Williams, 2007; Robbins and Finely, 1996). Commitment to transformational change will not happen without communication, and lots of it.

Uncertainty, a lack of information, and a sense that there were few opportunities to reduce uncertainty were common themes in all the studies.

Organizations fail to change when too many people believe they are not getting enough information about the changes. It may be impossible to meet everyone’s information needs. However, the need to know more is less disruptive when there are many opportunities for everyone to make sense of the changes. Without the entire organization participating in conversations about change, transformational change will not occur.

Local identification

When organizational members communicate during periods of intense change, they will generate organizing themes about identification. Self-concept is the organized set 339 of perceptions one has about one’s self (Cushman and Cahn, 1985). An aspect of self-concept is self-identity, and the organization of various self perceptions associated with organizational roles constitutes one’s organizational identity (Pratt and Foreman, 2000). Describing one’s self as female is part of one’s self-identity, but describing one’s self as a department head is part of one’s self-identity and also part of one’s organizational identity. A person may have multiple identities (Mead, 1934/1962), and multiple organizational identities (Cheney, 1991). For example, an organizational member may identify one’s self by one’s professional role, as part of a sub-unit, a unit, a department, a division, the company, or as a worker.

Individuals develop self-perceptions through interaction (Mead, 1934/1962), and organizational identification emerges in the communication members have with each other about each other. There are many ways members’ communication works to develop identification (Cheney, 1991; Lammers and Barbour, 2006; Scott, 2007). One avenue during change efforts is to develop a shared vision and another is to involve many in strategic planning processes (Robbins and Finely, 1996; Senge et al., 1999). Change will disrupt organizational identities, and members want to know what they will become and what the unit, division, or organization will become. Without communication that builds global and shared identification, members will resort to the older more local and independent identities.

Global distrust

During periods of intense change, organizational members will communicate about trust. Trust is an expectation, assumption, or belief of positive or non-negative outcomes that one can receive from another person’s future actions during uncertainty (Bhattacharya et al., 1998). Uncertainty implies vulnerability, and most contemporary definitions of trust include some belief in the positive intentions, behavior, or outcomes of another (Rousseau et al., 1998). Distrust is characterized by fear, skepticism, cynicism, and wariness (Lewicki et al., 1998). Mistrust, undefined in the literature, would be an inability to predict the value of engaging with another.

When organizational members distrust the agents of change or each other, strategic initiatives fail. Employees often distrust management during periods of planned change. A common way for members to express this distrust is to discuss organizational politics and the distrust members feel about how management might distribute resources.

Lack of productive humor

Humorous communication increases during intense organizational change. Humor is a form of communication that promotes laughter from discordant meanings or relationships (Duncan, 1982). Humorous communication works as a reframing mechanism (Wendt, 1998), and humor can be a norm and value as part of the culture (Trice and Beyer, 1993). Humor can be productive in the workplace by bringing social actors closer together, reducing stress, managing paradox, and building cohesiveness, but it can also be negative by being self-defeating, derisive, or part of anger (Geddes and Callister, 2007; Malone, 1980; Martin et al., 1993, 2003; McPherson, 2005; Romero and Cruthirds, 2006; Stacey, 1996). Organizational members can encourage or discourage change by how they use humor.

Poor interpersonal communication skills

The level of interpersonal communication skill will affect the direction of organizational change. Communication competence is an ability to accomplish goals with appropriate communication behaviors (Spitzberg and Cupbach, 1984).

Appropriateness refers to meeting the normative expectations of others in the social situation as well as using those behaviors most appropriate for the task at hand. Competence requires the performance of various communication skills and the perception of others that the performance was appropriate.

Three skills appear on most lists of communication skills related to competence:

  1. Responsiveness refers to those behaviors that attempt to understand the other and to communicate that understanding. These include verbal behaviors such as  paraphrasing, validating, and asking questions and nonverbal behaviors such as head nods, vocal encouragers, and back channeling.
  2. Openness refers to those behaviors an actor employs to improve the other’s understanding of the actor. Behaviors such as using personal language, being specific about experiences and feelings, and self disclosure may be part of openness.
  3. Flexibility is the ability to change communication behaviors in different situations. Being flexible means adjusting to different goals, tasks, people, and situations, and the competent communicator makes these adjustments in an appropriate way.

When members lack communication skills, communicating about change will be more difficult. Members will have difficulty making sense of change, feel greater uncertainty, identify less with the organization and its changes, and distrust others more.

Conflict avoidance

Intense change is a turbulent time, and the likelihood for conflict increases. Conflict is an expressed struggle over perceived differences (Folger et al., 2005). Individuals manage conflict in one of three general ways. Avoidance means never having to confront differences directly. Competitive tactics involve direct confrontations but may vary from argument about positions and ideas, to bids and counter offers, to verbal aggression and even violence. Integrative communication involves creating common goals, offering to help each other achieve individual goals, brainstorming to develop action plans, and creating common systems of accountability. People perceive integrative conflict communication as competent, competitive or controlling strategies as effective but inappropriate, and avoidant strategies as least competent (Gross et al., 2004).

In a time of intense organizational change, confronting differences is important. Conflict should exhibit a clash between newer conversational themes and older ones. Such conversations provide an opportunity to test strategic initiatives against older assumptions and expectations, and these conversations are the means for constructing emerging alternative identities, relationships, accounts, routines, and values (Griffin, 2002; Shaw, 2002). Members contrast emerging communication practices with older ones.

An inappropriate mix of loose and tight coupling

Getting to a bifurcation point capable of producing transformational change involves an accumulation of differences and a natural loose coupling of current behaviors. But when the system moves to a transformed state, it exhibits tighter coupling and the emergence of order from disorder. The development of some hierarchy of activity is common when systems emerge from transformational phase transitions such as the bifurcation points far from equilibrium (Barabasi, 2002). Decentralized structures may be best at initiating innovation and change, but there must be some centralization to implement (Rogers, 1995). The combination of factors noted above suggests organizational members may resist transformational change by loosening the couplings between each other as they cope with the initial disruptions of change and failing to construct tighter couplings as part of moving to a different set of routines and rituals.

Organizational members can decouple their system in three ways (Kingdon, 1973):

  1. Fragmentation is a process of decoupling goals. Fragmentation is a process of emphasizing local or individual goals at the expense of organizational wide segmentation is the global distrust, primarily of management. The distrust plays a role in the tendency to avoid conflict. Organizations experiencing dissociation and segmentation will have a difficult time accomplishing a unified effort. goals, and fragmentation is the last type of decoupling to occur.
  2. Dissociation is a process of decoupling horizontal units. There was evidence of dissociation in the tendencies to localize identities. Members identified with their local units and had little appreciation for other units or the whole.
  3. Segmentation is a process of decoupling vertically.

Sustaining transformational change involves the proper mix of loose and tight coupling.

Discussion

Communication during failed change efforts seldom involves enough communication opportunities, lacks any sense of emerging identification, engenders distrust, and lacks productive humor. These problems are compounded by conflict avoidance and a lack of interpersonal communication skills. Members’ communication decouples the system, sheltering the existing culture until it is safe for it to re-emerge later. No change in the intended direction is likely.

Results from this research point to the limitations of management communication and impersonal communication. Much of management literature assumes an exclusive place for management, as if managers were not a part of the organizations they manage. There is also the tendency to associate communication with the production of a message, as if finding the right words in the announced change would automatically bring commitment to the changes. Changing an organization’s culture is a task in and of itself, a task in addition to the tasks already going on as part of the routine business of an organization. Changing the communication practices of organizational members involves a give-and-take in which the change agents might change. Change is a messy business, and transformational change will not happen unless management is willing to tolerate the ambiguity and the sense that emerges in communication.

Results also reinforce the importance of communication skills in hiring practices. Communication occurs when two or more people in a social relationship create messages to make sense of the episodes they are creating. The process is inherently interpersonal. Hiring people with basic communication skills and training people in these skills not only improves the chances for sustaining a vibrant organization, but it also assists people in the rest of their lives as well.

References

Adler, M. (1967), The Difference in Man and the Difference it Makes, Holt Rinehart & Winston, New York, NY.

Agar, M. (2004), “An anthropological problem, a complex solution”, Human Organization, Vol. 63 No. 4, pp. 411-8.

Aldrich, H. and Ruef, M. (2006), Organizations Evolving, 2nd ed., Sage, Thousand Oaks, CA.

Argyris, C. (1992), On Organizational Learning, Blackwell, Cambridge, MA.

Barabasi, A. (2002), Linked: The New Science of Networks, Perseus Publishing, Cambridge, MA.

Bhattacharya, R., Devinney, T.M. and Pillutla, M.M. (1998), “A formal model of trust based on outcomes”, Academy of Management Review, Vol. 23, pp. 459-72.

Buckley, W. (1967), Sociology and Modern Systems Theory, Prentice-Hall, Englewood Cliffs, NJ.

Buckley, W. (1968), “Society as a complex adaptive system”, in Buckley, W. (Ed.), Modern Systems Research for the Behavioral Scientist, Aldine, Chicago, IL, pp. 490-513.

Buckley, W. (1998), Society: A Complex Adaptive System: Essays in Social Theory, Gordon and Breach Publishers, Amsterdam.

Cameron, K.S. and Quinn, R.E. (1999), Diagnosing Changing Organizational Culture: Based on the Competing Values Framework, Addison-Wesley, Reading, MA.

Cheney, G. (1991), Rhetoric in an Organizational Society: Managing Multiple Identities, University of South Carolina Press, Columbia, SC.

Clampitt, P.G. and Williams, M.L. (2007), “Decision downloading”, Sloan Management Review, Vol. 48 No. 2, pp. 77-82.

Cushman, D.P. and Cahn, D.D. (1985), Communication in Interpersonal Relationships, State University of New York Press, Albany, NY.

Daniels, T.D. and Spiker, B.K. (1983), “Social exchange and the relationship between information adequacy and relational satisfaction”, Western Journal of Speech Communication, Vol. 47, pp. 118-37.

Duncan, W.J. (1982), “Humor in management: prospects for administrative practice and research”, Academy of Management Review, Vol. 7, pp. 136-42.

Duran, R.L. (1983), “Communicative adaptability: a measure of social communicative competence”, Communication Quarterly, Vol. 31, pp. 320-6.

Folger, J.P., Poole, M.S. and Stutman, R.K. (2005), Working through Conflict: Strategies, Relationships, Groups, and Organizations, 5th ed., Longman, New York, NY.

Geddes, D. and Callister, R.R. (2007), “Crossing the line(s): a dual threshold model of anger in organizations”, Academy of Management Review, Vol. 32, pp. 721-46.

Gilbert, N. and Troitzsch, K.G. (2005), Simulation for the Social Scientist, Open University Press, New York, NY.

Griffin, D. (2002), The Emergence of Leadership: Linking Self-organization and Ethics, Routledge, London.

Gross, M.A., Guerro, L.K. and Alberts, J.K. (2004), “Perceptions of conflict strategies and communication competence in task-oriented dyads”, Journal of Applied Communication Research, Vol. 32, pp. 249-70.

Hernes, G. (1976), “Structural change in social processes”, American Journal of Sociology, Vol. 82, pp. 513-47.

Holland, J. (1995), Hidden Order: How Adaptation Builds Complexity, The Free Press, New York, NY.

Kauffman, S. (1993), The Origins of Order: Self Organization and Selection in Evolution, Oxford University Press, New York, NY.

Kauffman, S. (1995), At Home in the Universe: The Search for the Laws of Self-organization and Complexity, Oxford University Press, New York, NY.

Kingdon, M. (1973), Matrix Organization, Tavistock Press, London.

Lammers, J.C. and Barbour, J.B. (2006), “An institutional theory of organizational communication”, Communication Theory, Vol. 16, pp. 356-77.

Lewicki, R.J., McAllister, D.J. and Bies, R.J. (1998), “Trust and distrust: new relationships and realities”, Academy of Management Review, Vol. 23, pp. 439-58.

Lincoln, Y.S. and Guba, E.G. (1985), Naturalistic Inquiry, Sage, Newbury Park, CA.

McPherson, J. (2005), “Team humor during high-tech organizational change: subcultural integration, departmental differentiation, yet bureaucratic reproduction”, paper presented at the Annual Meeting of the International Communication Association Meeting in New York, NY.

Malone, P.B. (1980), “Humor: a double-edged tool for today’s managers?”, Academy of Management Review, Vol. 5, pp. 357-61.

Martin, R.A., Kupier, N.A., Olinger, L.J. and Dance, K.A. (1993), “Humor, coping with stress, self-concept, and psychological well-being”, Humor, Vol. 6 No. 4, pp. 89-104.

Martin, R.A., Puhlik-Doris, P., Larsen, G., Gray, J. and Weir, K. (2003), “Individual differences in uses of humor and their relation to psychological well-being: development of the humor styles questionnaire”, Journal of Research in Personality, Vol. 37 No. 1, pp. 48-75.

Mead, G.H. (1934/1962) in Morris, C.W. (Ed.), Mind, Self, and Society, University of Chicago Press, Chicago, IL.

Meyer, A.D., Goes, J.B. and Brooks, G.R. (1995), “Organizations reacting to hyperturbulence”, in Huber, G.P. and van de Ven, A.H. (Eds), Longitudinal Field Research Methods: Studying Processes in Organizational Change, Sage, Thousand Oaks, CA, pp. 66-111.

Miles, M.B. and Huberman, A.M. (1994), Qualitative Data Analysis: An Expanded Sourcebook, Sage, Thousand Oaks, CA.

Miller, C.C. and Cardinal, L.B. (1994), “Strategic planning and firm performance: a synthesis of more than two decades of research”, Academy of Management Journal, Vol. 37, pp. 1649-65.

Monge, P.R., Bachman, S.G., Dillard, J.P. and Eisenberg, E.M. (1982), “Communicator competence in the workplace: model testing and scale development”, in Burgoon, M. (Ed.), Communication Yearbook 5, Transaction, Brunswick, NJ, pp. 505-28.

Nadler, D.A., Shaw, R.B. and Walton, A.E. (1995), Discontinuous Change: Leading Organizational Transformation, Jossey-Bass, San Francisco, CA.

Polley, D. (1997), “Turbulence in organizations: new metaphors for organizational research”, Organization Science, Vol. 8 No. 5, pp. 445-57.

Pratt, M.G. and Foreman, P.O. (2000), “Clarifying managerial responses to multiple organizational identities”, Academy of Management Review, Vol. 25 No. 1, pp. 18-42.

Prigogine, I. and Stengers, I. (1984), Order out of Chaos: Man’s New Dialogue with Nature, Bantam, Toronto.

Rao, H. and Singh, J.V. (1999), “Types of variation in organizational populations: the speciation of organizational forms”, in Baum, J.A.C. and McKelvey, B. (Eds), Variations in Organization Science, Sage, Thousand Oaks, CA, pp. 63-77.

Rausch, G.A. (2005), “Beating the competition: the benefits of nonprofit coalition building”, paper presented at the Annual Meeting of the Western States Communication Association Meeting in San Francisco, CA, February.

Robbins, H. and Finely, M. (1996), Why Change doesn’t Work: Why Initiatives go Wrong and How to Try Again – and Succeed, Peterson’s, Princeton, NJ.

Rogers, E. (1995), Diffusion of Innovations, 4th ed., The Free Press, New York, NY.

Romero, E.J. and Cruthirds, K.W. (2006), “The use of humor in the workplace”, Academy of Management Perspectives, May, pp. 58-69.

Rousseau, D.M., Sitkin, S.B., Burt, R.S. and Camerer, C. (1998), “Not so different after all: a cross-discipline view of trust”, Academy of Management Review, Vol. 23, pp. 393-404.

Salem, P.J. (1999), “The changes and challenges for organizational communication in the next century”, in Salem, P. (Ed.), Organizational Communication and Change, Hampton Press, Cresskill, NJ, pp. 3-27.

Salem, P.J. (2004), “A longitudinal study of organizational communication climate”, paper presented at the Annual Meeting of the International Communication Association Meeting in New Orleans, LA, May.

Salem, P.J. (2007), “Making sense of knowledge management”, Vestnik, Vol. 5, pp. 47-68.

Salem, P.J. and Williams, M.L. (1984), “Uncertainty and satisfaction: the importance of types of information in hospital communication”, The Journal of Applied Communication Research, Vol. 12 No. 2, pp. 75-89.

Salem, P.J., Barclay, F. and Hoffman, M. (2003), “Organizational culture at the edge: a case study of organizational change”, paper presented at the Annual Meeting of the International Communication Association Meeting in San Diego, CA, May.

Scott, C.R. (2007), “Communication and social identity theory: existing and potential connections in organizational identification research”, Communication Studies, Vol. 58, pp. 123-38.

Senge, P., Kleiner, A., Roberts, C., Ross, R., Roth, G. and Smith, B. (1999), The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations, Currency Doubleday, New York, NY.

Shaw, P. (2002), Changing Conversations in Organizations: A Complexity Approach to Change, Routledge, London. Sheil, A. and Houser, M. (2003), “The unintended consequences of intentional conduct: a structurational analysis of organizational change”, paper presented to the National

Communication Association Convention in Miami, FL, November.

Smith, M.E. (2002), “Success rates for different types of organizational change”, Performance Improvement, Vol. 41 No. 1, pp. 26-33.

Spitzberg, B.H. and Cupbach, W.R. (1984), Interpersonal Communication Competence, Sage, Beverly Hills, CA.

Stacey, R.D. (1996), Complexity and Creativity in Organizations, Barrett-Koehler, San Francisco, CA.

Stacey, R.D. (2001), Complex Responsive Processes in Organizations: Learning and Knowledge Creation, Routledge, London.

Stacey, R.D. (2003), Strategic Management and Organizational Dynamics: The Challenge of Complexity, Prentice-Hall/Financial Times, New York, NY.

Thietart, R.A. and Forgues, B. (1995), “Chaos theory and organization”, Organization Science, Vol. 6 No. 1, pp. 19-31.

Trice, H.M. and Beyer, J.M. (1993), The Cultures of Work Organizations, Prentice-Hall, Englewood Cliffs, NJ.

Watzlawick, P., Weakland, J. and Fisch, R. (1974), Change: Principles of Problem Formation and Problem Resolution, Norton, New York, NY.

Weick, K.E. (1979), The Social Psychology of Organizing, 2nd ed., Addison-Wesley, Reading, MA.

Weick, K.E. (1995), Sensemaking in Organizations, Sage, Thousand Oaks, CA.

Weick, K.E. (2001), Making Sense of the Organization, Blackwell, Oxford.

Wendt, R.F. (1998), “The sound of one hand clapping: counterintuitive lessons extracted from paradoxes and double binds in a participative organization”, Management Communication Quarterly, Vol. 11, pp. 232-55.

Breaches of Health Information (US 2010 – 2015)

A research letter (Liu, Musen & Chou, 2015) published recently in the Journal of the American Medical Association1 described breaches of protected health information that had been reported from 2010 through 2013 by entities covered by the Health Insurance Portability and Accountability Act in the United States . Under the Health Information Technology for Economic and Clinical Health Act (2009), breaches involving the acquisition, access, use, or disclosure of protected health information and thus posing a significant risk to affected individuals must be reported.

We extend the original dataset of Liu et. al. to include breaches of health information up to the present. 2

Table 1 summarizes the number of incidents and victims of breaches of health information in the United States from January 2010 to August 2015, inclusive.

Counts and Victims of Health Information Breaches - US 2010-2015
Table 1. Number of incidents and victims of breaches of health information. † 2015 data are for January – August inclusive only.

The most striking feature is the fluctuation in the number of victims over time generally – and the tremendous spike in the number of victims in 2015 particularly.

Figure 1 depicts the distribution of victims/breach of health information as a series of boxplots.

Distribution of number of victims/incident (log scale) of breach of health information U.S. 2010-2015
Figure 1. Distribution of victims/incident (log scale) of breach of health information. † 2015 data are for January – August inclusive only.

We see that in seventy-five percent of all incidents, the number of victims/breach over the year has fallen consistently below 104 (10,000). A small number of incidents have involved 100,000 – 1,000,000 victims/breach, and an even smaller number have involved 1,000,000 – 10,000,000 victims/breach. Incidents involving more than 10,000,000 victims/breach made their first appearance in 2015.

Table 2 presents the Medians and Inter-Quartile Ranges of the distributions of victims/breach.

Median and IQR of Victims of Health Information Breaches - US 2010-2015
Table 2. First Quartile (Q1), Median, Third Quartile (Q3), and Inter-Quartile Range (IQR) of the distribution of victims/incident of breach of health information. † 2015 data are for January – August inclusive only.

The median number of victims of breaches of health is tending to increase over time, with a related increase in the dispersion of the number of victims/breach about the median.

Our focus in a few subsequent posts will be understanding the dynamics and implications of those breaches that have compromised the health information of 100,000+ patients.

Name Date Victims
Affinity Health Plan, Inc. 2010-04-14 344,579
Millennium Medical Management Resources, Inc. 2010-04-29 180,111
AvMed, Inc. 2010-06-03 1,220,000
Siemens Medical Solutions, USA, Inc 2010-06-04 130,495
Governor’s Office of Information Technology 2010-07-09 105,470
Iron Mountain Data Products, Inc. (now known as 2010-07-19 800,000
BlueCross BlueShield of Tennessee, Inc. 2010-11-01 1,023,209
Triple-S Management, Corp.; Triple-S Salud, Inc.; 2010-11-04 475,000
Medical Card System/MCS-HMO/MCS Advantage/MCS Life 2010-11-09 115,000
Ankle + Foot Center of Tampa Bay, Inc. 2011-01-03 156,000
Seacoast Radiology, PA 2011-01-10 231,400
GRM Information Management Services 2011-02-11 1,700,000
EISENHOWER MEDICAL CENTER 2011-03-30 514,330
Oklaholma State Dept. of Health 2011-04-11 132,940
IBM 2011-04-14 1,900,000
NA 2011-05-27 400,000
The Nemours Foundation 2011-10-07 1,055,489
Science Applications International Corporation (SA 2011-11-04 4,900,000
Sutter Medical Foundation 2011-11-17 943,434
Utah Department of Technology Services 2012-04-11 780,000
Emory Healthcare 2012-04-18 315,000
South Carolina Department of Health and Human Services 2012-04-24 228,435
Memorial Healthcare System 2012-08-16 105,646
Alere Home Monitoring, Inc 2012-10-18 116,506
Crescent Health Inc. – a Walgreens Company 2013-02-22 109,000
Digital Archive Management 2013-05-07 189,489
RCR Technology Corporation 2013-07-01 187,533
Shred-it International Inc. 2013-07-11 277,014
Advocate Health and Hospitals Corporation, d/b/a Advocate Medical Group 2013-08-23 4,029,530
AHMC Healthcare Inc. and affiliated Hospitals 2013-10-25 729,000
Horizon Healthcare Services, Inc 2014-01-03 839,711
Triple-C, Inc. 2014-01-24 398,000
St. Joseph Health System 2014-02-05 405,000
Indian Health Service 2014-04-01 214,000
Sutherland Healthcare Solutions, Inc. 2014-05-22 342,197
Montana Department of Public Health and Human Services 2014-07-07 1,062,509
Community Health Systems Professional Services Corporation 2014-08-20 4,500,000
Xerox State Healthcare, LLC 2014-09-10 2,000,000
Touchstone Medical Imaging, LLC 2014-10-03 307,528
Walgreen Co. 2014-12-15 160,000
Georgia Department of Community Health 2015-03-02 557,779
Georgia Department of Community Health 2015-03-02 355,127
Virginia Department of Medical Assistance Services (VA-DMAS) 2015-03-12 697,586
Anthem, Inc. Affiliated Covered Entity 2015-03-13 78,800,000
Premera Blue Cross 2015-03-17 11,000,000
Advantage Consolidated LLC 2015-03-18 151,626
CareFirst BlueCross BlueShield 2015-05-20 1,100,000
Beacon Health System 2015-05-22 306,789
University of California, Los Angeles Health 2015-07-17 4,500,000
Medical Informatics Engineering 2015-07-23 3,900,000
Empi Inc and DJO, LLC 2015-08-20 160,000

 

  1.  Liu V, Musen MA, Chou T. Data Breaches of Protected Health Information in the United States. JAMA. 2015;313(14):1471-1473. doi:10.1001/jama.2015.2252.
  2. Our source of data is the Breach Portal: Notice to the Secretary of HHS Breach of Unsecured Protected Health Information, Office for Civil Rights, U.S. Department of Health and Human Services, accessed at https://ocrportal.hhs.gov/ocr/breach/breach_report.jsf on September 1, 2015.

Update of Service-Dominant Logic – 2015

In their recent paper “Institutions and axioms: an extension and update of service-dominant logic,” Stephen Vargo and Robert Lusch summarize key steps in the evolution of Service-Dominant Logic since their original formulation of S-DL appeared in 2004.

Within this one paper, Vargo and Lusch provide several perspectives on the evolutionary path of S-DL.

Motivation

Along its evolutionary path, Service-Dominant logic has recognized and taken steps to address two deficiencies:

  1. An imprecision in delineation of the Foundational Premises (FPs) and specification of the Axioms of S-DL
  2. An absence of a clearly articulated specification of the mechanisms of (often massive-scale) co-ordination and co-operation involved in the co-creation of value through markets and, more broadly, in society.

Present Contribution

To alleviate this limitation and facilitate a better understanding of co-operation (and co-ordination), an eleventh Foundational Premise (Fifth Axiom) is introduced, focusing on:

  • the role of institutions and
  • institutional arrangements
  • in systems of value co-creation (aka service ecosystems)

 

While Vargo and Lusch have collaborated together and separately with many other researchers in the past decade, their tradition has been to come together (on their own) every two years to refine their specification of Service-Dominant Logic (S-DL):

2004
Evolving to a new dominant logic for marketing
The four service marketing myths remnants of a goods-based, manufacturing model
2006
Service-dominant logic
Service-dominant logic: reactions, reflections and refinements
Service-Dominant Logic as a foundation for a general theory
2008
Service-dominant logic: continuing the evolution
Why “service”?
The service-dominant mindset
From goods to service (s): Divergences and convergences of logics
Service-Dominant Logic, market theory and marketing theory
A service logic for service science
2010
SD logic: accommodating, integrating, transdisciplinary
“Relationship” in Transition: An Introduction to the Special Issue on Relationship and Service-Dominant Logic
From repeat patronage to value co-creation in service ecosystems: a transcending conceptualization of relationship
2012
The nature and understanding of value: a service-dominant logic perspective
Gaining competitive advantage with service-dominant logic
The forum on markets and marketing (FMM) Advancing service-dominant logic
2014
Service-dominant logic: Premises, perspectives, possibilities
An introduction to service-dominant logic
Inversions of service-dominant logic
The service-dominant logic of marketing: Dialog, debate, and directions (Book)
Service-Dominant logic as a foundation for a general theory

Table 1. Bi-annual collaboration of Vargo-Lusch on Service-Dominant Logic (2004 – 2014).

Vargo and Lusch have represented the evolution of Service-Dominant Logic in terms of refinements of its Foundational Premises (FPs) and Axioms:

Foundational
Premise
2004 2008 2015
FP1 The application of specialized skills and knowledge is the fundamental unit of exchange. Service is the fundamental basis of exchange No Change – AXIOM STATUS
FP2 Indirect exchange masks the fundamental unit of exchange. Indirect exchange masks the fundamental basis of exchange. No Change
FP3 Goods are distribution mechanisms for service provision. No Change No Change
FP4 Knowledge is the fundamental source of competitive advantage. Operant resources are the fundamental source of competitive advantage. Operant resources are the fundamental source of strategic benefit.
FP5 All economies are service economies. No Change No Change
FP6 The customer is always the co-producer. The customer is always a co-creator of value. Value is co-created by multiple actors, always including the beneficiary – AXIOM STATUS
FP7 The enterprise can only make value propositions. The enterprise cannot deliver value, but only offer value propositions. Actors cannot deliver value but can participate in the creation and offering of value propositions.
FP8 Service-centered view is customer oriented and relational. A service-centered view is inherently customer-oriented and relational. A service-centered view is inherently beneficiaryoriented and relational.
FP9 All social and economic actors are resource integrators. No change – AXIOM STATUS
FP10 Value is always uniquely and phenomenologically determined by the beneficiary. No change. – AXIOM STATUS
FP11 Value co-creation is coordinated through actor-generated institutions and institutional arrangements. – AXIOM STATUS

Table 2. Development of the Foundational Premises of Service-Dominant Logic.

At a more general level of description, we would describe the evolution of Service-Dominant Logic turns on promoting three new classes of objects:

  • actor
  • institution
  • service ecosystem

… and four dynamics of service:

  • the role of co-operation (versus competition) in service provision
  • the role of institutions in value co-creation
  • the role of experience in service evaluation
  • the role of value co-creation in service innovation

The Service-Dominant Logic of today imagines resource-integrating, reciprocal-service-providing actors co-create value through meaning-laden experiences in service ecosystems governed and evaluated through institutional arrangements.

The major components of this emerging narrative are presented in Figure 1.

Vargo and Lusch 2015 - Fig 1 The narrative and process of S-D Logic

Figure 1. The narrative and process of Service-Dominant Logic.

We’d now like to consider the innovative elements of this narrative, beginning with the three new classes of objects (Actor, Institution, and Service Ecosystem) in Service-Dominant Logic.

References

Abernathy, W. J., & Clark, K. (1985). Innovation: mapping the winds of creative destruction. Research Policy, 14(1), 3–22.

Akaka, M. A., Vargo, S. L., & Lusch, R. F. (2013). The complexity of context: a service ecosystems approach for international marketing. Journal of International Marketing, 21(4), 1–20.

Alderson, W. (1957). Marketing behavior and executive action. Homewood: Richard D. Irwin.

Alderson, W. (1965). Dynamic marketing behavior. Homewood: Richard D. Irwin.

Alderson, W., & Cox, R. (1948). Towards a theory of marketing. Journal of Marketing, 8(2), 137–152.

Araujo, L., & Spring, M. (2006). Services, products, and the institutional structure of production. Industrial Marketing Management, 35(7), 797–805.

Arndt, J. (1981). The political economy of marketing systems: reviving the institutional approach. Journal of Macromarketing, 1(2), 36–47.

Arnould, E. J. (2006). Service-dominant logic and consumer culture theory: natural allies in and emerging paradigm. Marketing Theory, 6(3), 293–298.

Arrow, K. J. (1987). Reflections on the essays. In G. Feiwel (Ed.), Arrow and the foundations of the theory of economic policy (pp. 727–34). NY: NYU Press.

Arthur, B. W. (2009). The nature of technology: What it is and how it evolves. New York: Free Press.

Arthur, B. W. (2013). Complexity economics: A different framework for economic thought. Santa Fe Institute Working Paper 2013-04-012. Santa Fe: Santa Fe Institute.

Arthur, B. W. (2014). Complexity and the economy. Oxford: Oxford University Press.

Battilana, J., & D’Aunno, T. (Eds.). (2009). Institutional work and the paradox of embedded agency. New York: Cambridge University Press.

Beinhocker, E. D. (2006). The origins of wealth: evolution, complexity, and the radical remaking of economics. Boston: Harvard Business School Press.

Beinhocker, E. D. (2010). Evolution as computation: Implications for economic theory and ontology. Santa Fe Working Paper 2010-12037. Santa Fe: Santa Fe Institute.

Bello, D. C., Lohtia, R., & Sangtani, V. (2004). An institutional analysis of supply chain innovations in global marketing channels. Industrial Marketing Management, 33(1), 57–64.

Bernstein, W. J. (2004). The birth of plenty: How the prosperity of the world was created. New York: McGraw-Hill.

Bettencourt, L., Lusch, R. F., & Vargo, S. L. (2014). A service lens on value creation: how marketing should enable strategic advantage. California Management Review, 57(1), 44–66.

Bill, J. A., & Hardgrave, R. L., Jr. (1981). Comparative politics: The Quest for theory. Washington, DC: Bell and Howell.

Bourdieu, P. (1977). Outline of a theory of practice. Cambridge: The Press Syndicate of the University of Cambridge.

Brodie, R. J., Glynn, M. S., & Little, V. (2006). The service brand and the service-dominant logic: missing fundamental premise or the need for stronger theory? Marketing Theory, 6(3), 363–379.

Brodie, R. J., Saren, M., & Pels, J. (2011). Theorizing about the service dominant logic: the bridging role of middle range theory. Marketing Theory, 11(March), 175–191.

Cannon, J. P., Achrol, R. S., & Gundlach, G. T. (2000). Contracts, norms, and plural form governance. Journal of the Academy of Marketing Science, 28(2), 180–194.

Carson, S. J., Devinney, T. M., Dowling, G. R., & John, G. (1999). Understanding institutional designs within marketing value systems. The Journal of Marketing, 63(Special Issue), 115–130.

Chandler, J., & Lusch, R. F. (2015). Service systems: a broadened framework and research agenda on value propositions, engagement and service experience. Journal of Service Research, 18(1), 6–22.

Chandler, J., & Vargo, S. L. (2011). Contextualization: network intersections, value-in-context, and the co-creation of markets. Marketing Theory, 11(1), 35–49.

Coase, R. H. (1937). The nature of the firm. Economica, 4(16), 386–405. Coase, R. H. (1972). Industrial organization: A proposal for research. V. Fuchs (ed.) in Policy Issues and Research Opportunities in Industrial Organization: National Bureau of Economic Research (pp. 59–73).

Commons, J. R. (1934). Institutional economics: its place in political economy. New York: Macmillan.

Davis, L. E., & North, D. C. (1971). Institutional change and American economic growth. Cambridge: Cambridge University Press.

DiMaggio, P. J. (1988). Interest and agency in institutional theory. In L. Zucker (Ed.), Institutional patterns and organizations: Culture and environment (pp. 3–21). Cambridge: Ballinger.

DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147–160.

DiMaggio, P. J., & Powell, W. W. (Eds.). (1991). BIntroduction^. The New Institutionalism in Organizational Analysis (pp. 1–38). Chicago: University of Chicago Press.

Duddy, E. A., & Revzan, D. A. (1953). Marketing: An institutional approach. New York: McGraw-Hill.

Durkheim, E. (1912/2008). The elementary forms of the religious life [1912]. Oxford World’s Classics.

Edvardsson, B., Tronvol, B., & Gruber, T. (2011). Expanding understanding of service exchange and value co-creation: a social construction approach. Journal of the Academy of Marketing Science, 39(2), 327–39.

Friedland, R., & Alford, R. R. (1991). Bringing society back in: Symbols, practices and institutional contradictions. In P. DiMaggio & W. W. Powell (Eds.), The New Institutionalism in Organizational Analysis (pp. 232–263). Chicago: University of Chicago Press.

Giddens, A. (1984). The constitution of society. Berkeley: University of California Press.

Giesler, M. (2008). Conflict and compromise: drama in marketplace evolution. Journal of Consumer Research, 34(April), 739–753.

Granovetter, M. (1985). Economic action and social structure: the problem of embeddedness. American Journal of Sociology, 91(3), 481– 510.

Grewal, R., & Dharwadkar, R. (2002). The role of the institutional environment in marketing channels. The Journal of Marketing, 66(3), 82–97.

Gronroos, C. (2008). Service logic revisited: who creates value? And who co-creates? European Business Review, 20(40), 298–314.

Gronroos, C., & Voima, P. (2013). Critical service logic: making sense of value creation and co-creation. Journalofthe AcademyofMarketing Science, 41(2), 133–150.

Gundlach, G. T., & Achrol, R. S. (1993). Governance in exchange: contract law and its alternatives. Journal of Public Policy & Marketing, 12(2), 141–155.

Hakansson, H., Ford, D., Gadde, L. E., Snehota, I., & Waluszewski, A. (2009). Business in networks. Chichister: John Wiley and Sons.

Harrison, D., & Kjellberg, H. (2010). Segmenting a market in the making: Industrial market segmentation as construction. Industrial Marketing Management, 39(5), 784–792.

Hawley, A. H. (1968). Human ecology. In D. L. Sills (Ed.), International encyclopedia of the social sciences (pp. 328–337). New York: Macmillan.

Heide, J. B., & John, G. (1992). Do norms matter in marketing relationships? The Journal of Marketing, 56(2), 32–44.

Hinings, C., Tolbert, P. S., Greenwood, R., & Oliver, C. (2008). Organizational institutionalism and sociology: A reflection. The Sage handbook of organizational institutionalism (pp. 473–490).

Holland, J. (2014). Complexity: A very short introduction. Oxford: Oxford University Press.

Humphreys, A. (2010). Megamarketing: the creation of markets as a social process. Journal of Marketing, 74, 1–19.

Hunt, S. D. (1983). General theories and the fundamental explananda of marketing. Journal of Marketing, 47, 9–17.

Hunt, S. D. (2012). Toward the institutionalization of macromarketing. Journal of Macromarketing, 32(4), 404–411.

Kates, S. M. (2004). The dynamics of brand legitimacy: an interpretive study in the gay men’s community. Journal of Consumer Research, 31(2), 455–464.

Kjellberg, H., & Helgesson, C.-F. (2006). Multiple versions of markets: multiplicity and performativity in market practice. Industrial Marketing Management, 35(7), 839–855.

Kjellberg, H., & Helgesson, C.-F. (2007). On the nature of markets and their practices. Marketing Theory, 7(2), 137–162.

Koestler, A. (1973). The tree and the candle. In W. Gray & N. D. Rizzo (Eds.), Unity through Diversity, pt I (pp. 287–314). New York: Gordon and Breach Science Publishers.

Korkman, O., Storbacka, K., & Harald, B. (2010). Practices as markets: value co-creation in e- invoicing. Australasian Marketing Journal, 18(4), 236–247.

Latour, B. (2005). Reassembling the Social: An introduction to actornetwork-theory. Oxford: Oxford University Press.

Lawrence, T. B., & Suddaby, R. (Eds.). (2006). Institutions and institutional work. London: Sage.

Lawrence, T. B., Suddaby, R., & Leca, B. (2009). Introduction: Theorizing and studying institutional work. In T. B. Lawrence, R. Suddaby, & B. Leca (Eds.), Institutional work: Actors and agency in institutional studies of organizations (pp. 1–28). New York: Cambridge University Press.

Layton, R. A. (2011). Towards a theory of marketing systems. European Journal of Marketing, 45(1/2), 259–267.

Lewin, K. (1939). Field theory and experiment in social psychology. American Journal of Sociology, 44(6), 868–896.

Loasby, B. J. (2000). Market institutions and economic evolution. Journal of Evolutionary Economics, 10(3), 297–309.

Loasby, B. J. (2001). Knowledge, institutionsand evolution ineconomics. New York: Routledge.

Lusch, R. F., & Brown, J. R. (1996). Interdependency, contracting, and relational behavior in marketing channels. Journal of Marketing, 60, 19–38.

Lusch, R. F., & Nambisan, S. (2015). Service innovation: a servicedominant (s-d) logic perspective. Management Information Systems Quarterly, 39(1), 155–175.

Lusch, R. F., & Tay, N. (2004). Agent-based modeling: Gaining insight into firm and industry performance. In C. Moorman & D. R. Lehman (Eds.), Assessing marketing strategy performance (pp. 213–227). Cambridge: Marketing Science Institute.

Lusch, R. F., & Vargo, S. L. (2006). Service-dominant logic as a foundation for a general theory. In R. F. Lusch & S. L. Vargo (Eds.), The service-dominant logic of marketing: Dialog, debate, and directions (pp. 381–420). Armonk: ME Sharpe.

Lusch, R. F., & Vargo, S. L. (2014). Service-dominant logic: Premises, perspectives, possibilities. Cambridge: Cambridge University Press.

Lusch, R. F., & Webster, F. E. (2011). A stakeholder-unifying, cocreation philosophy for marketing. Journal of Macromarketing, 31(2), 129– 134.

Lusch, R. F., Vargo, S. L., & O’Brien, M. (2007). Competing through service: insights from service-dominant logic. Journal of Retailing, 83(1), 5–18.

Lusch, R. F., Vargo, S. L., & Tanniru, M. (2010). Service, value-networks, and learning. Journal of the Academy of Marketing Science, 38(1), 19–31.

Macneil, I. R. (1980). The new social contract: An inquiry into modern contractual relations. London: Yale University Press.

Maglio, P., Vargo, S. L., Caswell, N., & Spohrer, J. (2009). The service system is the basic abstraction of service science. Information Systems and e-Business Management, 7(4), 395–406.

McAlexander, J. H., Dufault, B. L., Martin, D. M., & Schouten, J. W. (2014). The marketization of religion: field, capital, and consumer identity. Journal of Consumer Research, 41(3), 858–875.

McColl-Kennedy, J. R., Vargo, S. L., Dagger, T. S., Sweeney, J. C., & van Kasteren, Y. (2012). Health care customer value co-creation practice styles. Journal of Service Research, 15(November), 370–89.

Menard, C. (1996). Markets as institutions versus organizations as markets? Disentangling some fundamental concepts. Journal of Economic Behavior and Organization, 28(2), 161–82.

Merriam Webster Online: An encyclopedia of the Britannica Co. (2015) http://www.merriam-webster.com/dictionary/interaction.

Merz, M., He, Y., & Vargo, S. L. (2009). The evolving brand logic: a service-dominant logic perspective. Journal of the Academy of Marketing Science, 37(3), 338–44.

Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: formal structure as myth and ceremony. American Journal of Sociology, 83(2), 340–363.

Mitchell, W. C. (1937). The backward art of spending money and other essays. New York: Augustus M. Kelley, Inc.

Mitchell, M. (2009). Complexity: A guided tour. New York: Oxford University Press.

Moeller, S. (2008). Customer integration – a key to an implementation perspective of service provision. Journal of Service Research, 11(November), 197–210.

Mokyr, J. (2002). The gifts of Athena: Historical origins of the knowledge economy. P