Navigation
Topics
Log In
Log Out
:
Special Search
New Today
New This Week
Advanced Search
Tree View
Your Account
Edit Profile
Register
Forgot Password
Tools
Help/Instructions
Policies
...
|
| Professional IT auditing standards --... |
|
| Author |
Message |
    
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 268
Registered: 12-2007
Best of Black Box? 
Votes: 2 (A keeper?) | | Posted on Thursday, February 28, 2008 - 9:27 am: | 
|
Some of us are in states where audits have been mandated or contemplated. In CT, it is my opinion that the voting machine audit procedure, as of February 2008, is neither an audit nor does it provide a sufficient amount of information that has further value if the audit were to be escalated (and so far the results have not been escalated, in my understanding).
If you have an audit, or are giving input in a state that is considering "audits", it's important to insist that the audit be meaningful, and examine meaningful issues, rather than be a poorly designed marketing tool to promote electronic voting without seriously examining any aspect of it.
When I started looking at CT's procedures, it took me a long time to articulate what was very wrong with them (and so far, I have not yet articulated that to my state's GAE committee that writes election legislation, but I will.)
However, because I am not an auditor, I was at a loss as to what might be done instead, and what would constitute meaningful areas to examine.
I was excited today to surf upon the attached document, because in reading how a professional IT audit is designed and the standards for its conduct, the shortcomings of our audit and the areas that in fact would be productive to audit became much clearer.
Here's the document I found. I make no claims as to its ultimate excellence, and I know nothing of the reputation of the organization that created it. I only know that it has provided me a leg up in gaining insight and identifying far more constructive and productive alternatives to the "audit" we have in CT now.
I invite anyone interested to skim and selective read the document and add your ideas and thoughts in the thread. If lurking is your preference, I hope you find this a useful touchpoint in your own state.
I remind myself that however poorly created, it's still worth observing election audits because they are a chance to observe an ROV in action doing his/her job, and to see firsthand the types of discrepancies, problems, and also excellence in handling them (or challenges in handling them). As information/orientation opportunities, imho they have value and are worth taking while pushing for something better in the way of monitoring our election administration systems.
http://www.isaca.org/AMTemplate.cfm?Section=Standards2&Template=/ContentManageme nt/ContentDisplay.cfm&ContentID=39354 |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4689
Registered: 12-2004
Best of Black Box?
Votes: 3 (A keeper?) | | Posted on Thursday, February 28, 2008 - 1:01 pm: |
|
I have pulled out the standards given at the beginning of each section. Bold is mine.
"IS" refers to Information Systems. As you read through the following excerpts, mentally change "IS" or "Information Systems" to "Election Systems"
In section S16, mentally change "E-Commerce" to "Election or Voter Data Transfer/Processing by Electronic Means"
S1 Audit Charter
03 The purpose, responsibility, authority and accountability of the information systems audit function or information systems audit assignments should be appropriately documented in an audit charter or engagement letter.
04 The audit charter or engagement letter should be agreed and approved at an appropriate level within the organisation(s).
S2 Independence
03 Professional Independence
In all matters related to the audit, the IS auditor should be independent of the auditee in both attitude and appearance.
04 Organisational Independence
The IS audit function should be independent of the area or activity being reviewed to permit objective completion of the audit assignment.
S3 Professional Ethics and Standards
05 The IS auditor should adhere to the ISACA Code of Professional Ethics in conducting audit assignments.
06 The IS auditor should exercise due professional care, including observance of applicable professional auditing standards, in conducting the audit assignments.
S4 Professional Competence
03 The IS auditor should be professionally competent, having the skills and knowledge to conduct the audit assignment.
04 The IS auditor should maintain professional competence through appropriate continuing professional education and training.
S5 Planning
03 The IS auditor should plan the information systems audit coverage to address the audit objectives and comply with applicable laws and professional auditing standards.
04 The IS auditor should develop and document a risk-based audit approach.
05 The IS auditor should develop and document an audit plan that lists the audit detailing the nature and objectives, timing and extent, objectives and resources required.
06 The IS auditor should develop an audit program and/or plan and detailing the nature, timing and extent of the audit procedures required to complete the audit.
S6 Performance of Audit Work
03 Supervision—IS audit staff should be supervised to provide reasonable assurance that audit objectives are accomplished and applicable professional auditing standards are met.
04 Evidence—During the course of the audit, the IS auditor should obtain sufficient, reliable and relevant evidence to achieve the audit bjectives. The audit findings and conclusions are to be supported by appropriate analysis and interpretation of this evidence.
05 Documentation—The audit process should be documented, describing the audit work performed and the audit evidence that supports supporting the IS auditor's findings and conclusions.
S7 Reporting
03 The IS auditor should provide a report, in an appropriate form, upon completion of the audit. The report should identify the organisation, the intended recipients and any restrictions on circulation.
04 The audit report should state the scope, objectives, period of coverage and the nature, timing and extent of the audit work performed.
05 The report should state the findings, conclusions and recommendations and any reservations, qualifications or limitations in scope that the IS auditor has with respect to the audit.
06 The IS auditor should have sufficient and appropriate audit evidence to support the results reported.
07 When issued, the IS auditor’s report should be signed, dated and distributed according to the terms of the audit charter or engagement letter.
S8 Follow-Up Activities
03 After the reporting of findings and recommendations, the IS auditor should request and evaluate relevant information to conclude whether appropriate action has been taken by management in a timely manner.
S9 Irregularities and Illegal Acts
03 In planning and performing the audit to reduce audit risk to a low level, the IS auditor should consider the risk of irregularities and illegal acts.
04 The IS auditor should maintain an attitude of professional skepticism during the audit, recognising the possibility that material misstatements due to irregularities and illegal acts could exist, irrespective of his/her evaluation of the risk of irregularities and illegal acts.
05 The IS auditor should obtain an understanding of the organisation and its environment, including internal controls.
06 The IS auditor should obtain sufficient and appropriate audit evidence to determine whether management or others within the organisation have knowledge of any actual, suspected or alleged irregularities and illegal acts.
07 When performing audit procedures to obtain an understanding of the organisation and its environment, the IS auditor should consider unusual or unexpected relationships that may indicate a risk of material misstatements due to irregularities and illegal acts.
08 The IS auditor should design and perform procedures to test the appropriateness of internal control and the risk of management override of controls.
09 When the IS auditor identifies a misstatement, the IS auditor should assess whether such a misstatement may be indicative of an irregularity or illegal act. If there is such an indication, the IS auditor should consider the implications in
relation to other aspects of the audit and in particular the representations of management.
10 The IS auditor should obtain written representations from management at least annually or more often depending on the audit engagement. It should:
• Acknowledge its responsibility for the design and implementation of internal controls to prevent and detect irregularities or illegal acts
• Disclose to the IS auditor the results of the risk assessment that a material misstatement may exist as a result of an irregularity or illegal act
• Disclose to the IS auditor its knowledge of irregularities or illegal acts affecting the organisation in relation to:
– Management
– Employees who have significant roles in internal control
• Disclose to the IS auditor its knowledge of any allegations of irregularities or illegal acts, or suspected irregularities or illegal acts affecting the organisation as communicated by employees, former employees, regulators and others
11 If the IS auditor has identified a material irregularity or illegal act, or obtains information that a material irregularity or illegal act may exist, the IS auditor should communicate these matters to the appropriate level of management in a timely manner.
12 If the IS auditor has identified a material irregularity or illegal act involving management or employees who have significant roles in internal control, the IS auditor should communicate these matters in a timely manner to those charged with governance.
13 The IS auditor should advise the appropriate level of management and those charged with governance of material weaknesses in the design and implementation of internal control to prevent and detect irregularities and illegal acts that may have come to the IS auditor’s attention during the audit.
14 If the IS auditor encounters exceptional circumstances that affect the IS auditor’s ability to continue performing the audit because of a material misstatement or illegal act, the IS auditor should consider the legal and professional responsibilities applicable in the circumstances, including whether there is a requirement for the IS auditor to report to those who entered into the engagement or in some cases those charged with governance or regulatory authorities or consider withdrawing from the engagement.
15 The IS auditor should document all communications, planning, results, evaluations and conclusions relating to material irregularities and illegal acts that have been reported to management, those charged with governance, regulators and others.
S10 IT Governance
03 The IS auditor should review and assess whether the IS function aligns with the organisation’s mission, vision, values, objectives and strategies.
04 The IS auditor should review whether the IS function has a clear statement about the performance expected by the business (effectiveness and efficiency) and assess its achievement.
05 The IS auditor should review and assess the effectiveness of IS resource and performance management processes.
06 The IS auditor should review and assess compliance with legal, environmental and information quality, and fiduciary and security requirements.
07 A risk-based approach should be used by the IS auditor to evaluate the IS function.
08 The IS auditor should review and assess the control environment of the organisation.
09 The IS auditor should review and assess the risks that may adversely effect the IS environment.
S11 Use of Risk Assessment in Audit Planning
03 The IS auditor should use an appropriate risk assessment technique or approach in developing the overall IS audit plan and in determining priorities for the effective allocation of IS audit resources.
04 When planning individual reviews, the IS auditor should identify and assess risks relevant to the area under review.
S12 Audit Materiality
03 The IS auditor should consider audit materiality and its relationship to audit risk while determining the nature, timing and extent of audit procedures.
04 While planning for audit, the IS auditor should consider potential weakness or absence of controls and whether such weakness or absence of control could result into significant deficiency or a material weakness in the information system.
05 The IS auditor should consider the cumulative effect of minor control deficiencies or weaknesses and absence of controls to translate into significant deficiency or material weakness in the information system.
06 The report of the IS auditor should disclose ineffective controls or absence of controls and the significance of the control deficiencies and possibility of these weaknesses resulting in a significant deficiency or material weakness.
13 Using the Work of Other Experts
03 The IS auditor should, where appropriate, consider using the work of other experts for the audit.
04 The IS auditor should assess and be satisfied with the professional qualifications, competencies, relevant experience, resources, independence and quality control processes of other experts, prior to engagement.
05 The IS auditor should assess, review and evaluate the work of other experts as part of the audit and conclude the extent of use and reliance on expert’s work.
06 The IS auditor should determine and conclude whether the work of other experts is adequate and complete to enable the IS auditor to conclude on the current audit objectives. Such conclusion should be clearly documented.
07 The IS auditor should apply additional test procedures to gain sufficient and appropriate audit evidence in circumstances where the work of other experts does not provide sufficient and appropriate audit evidence.
08 The IS auditor should provide appropriate audit opinion and include scope limitation where required evidence is not obtained through additional test procedures.
S14 Audit Evidence
03 The IS auditor should obtain sufficient and appropriate audit evidence to draw reasonable conclusions on which to base the audit results.
04 The IS auditor should evaluate the sufficiency of audit evidence obtained during the audit.
S15 IT Controls
03 The IS auditor should evaluate and monitor IT controls that are an integral part of the internal control environment of the organisation.
04 The IS auditor should assist management by providing advice regarding the design, implementation, operation and improvement of IT controls.
S16 E-Commerce
03 The IS auditor should evaluate applicable controls and assess risk when reviewing e-commerce environments to ensure that e-commerce transactions are properly controlled.
Reproduction of selections of this publication, for internal and non-commercial or academic use only, is permitted and must include full attribution as follows: "© 2008 ISACA. This document is reprinted with the permission of ISACA." |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4690
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, February 28, 2008 - 1:15 pm: |
|
Christine, this is an invaluable resource for understanding what an audit is supposed to be.
It is important to start using this word properly in relation to elections.
No wonder election integrity folks who have been attempting to carry out some of these functions have been frequently greeted with resistance from election staff. As described in these standards, auditing must be carried out with an attitude of skepticism in order to be meaningful. Election officials then interpret this as a personal attack, rather than as an auditor doing their job appropriately.
If everyone had prior agreement about what was being done and why this would eventually change everyone's attitude--or risk that any obfuscation or obstruction would be a red flag that the auditor would be required to report. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 271
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, February 28, 2008 - 1:35 pm: |
|
Glad you think it's valuable. It helps to move the conversation out of ad hominem characterizations of activists into professional standards that can HARDLY be attacked as radical, tin foil, etc. (grin -- I bet Bev is thinking: 1) they can and 2) they will!). I like that possibility anyhow. 
Although maybe this could apply generally to what a good audit might look like, because of the plethora of half baked, amateurish "audits" floating around, I had in mind looking at how to create any MEANINGFUL evaluation of the IT piece of an election administration system.
The one that really has me -- what? gobsmacked? - is that ROVs are responsible for the counting device, while being responsible for the registration info's accuracy, while being responsible for reporting the election results.
If I understand separation of duties in a traditional organization, that would not fly -- they would imho be deemed "incompatible duties".
You have the potential to have an IT administrator who can e.g. erase t he memory card and maybe change certain programming parameters , who also reports the vote count, who also controls "how many people are registered and eligible to vote" info.
Am I missing something here? Please straighten me out if I am, people.
(Message edited by ctwatcher on February 28, 2008) |
Jean Braun
Voting Rights Forum Participant
Username: Jean_b
Post Number: 1
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Sunday, March 9, 2008 - 6:41 pm: |
|
These guidelines are great for a professional audit. However, the kind of audit I can see for election counting probably needs quotes around the word - "Audit" e.g.
}}}On the principle that the more people observing the counting, the more difficult to enact fraud, providing for and cultivating a national habit of counting the paper record - which should be the official record - at every precinct, under public observation, immediately after polls close, with that count being posted at the precinct, could be the best safeguard against ballots being altered anywhere else along the chain.
It would, of course, be different from a professional audit, but would be just as tamper-proof. Many people don't have the time or inclination to do a lot of traveling on election day, but dropping in at the precinct would be easy. Effectively, it would be an audit - a public count of the ballots.
With the narrowing of precinct locations, as in Ohio this past election, logiistical and other preparations would need to be accommodated, but the neighborhood precinct count is very doable.
An auxiliary to this is the need for each ballot to have an unique ID, a practice which has been dropped in Ohio, at least, over the last few years. If each voter knew the ID of his/her ballot, it would be a means of verifying that the election ballot which is counted is the one (s)he cast. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 359
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 10, 2008 - 12:03 pm: |
|
The ballot ID question will continue to be problematic due to the secret ballot issues.
The idea of larger precincts due to machine counting is thrilling to those who see everything about elections as a money saving event.
If I may, Jean, I'd like to suggest that a hot count after the polls close not be viewed as an audit -- to prove the machine did okay, but as an actual vote count, for which the MACHINE is the corroborating, but not final, word. The paper ballots should be IT.
I have adopted a personal code about my use of language around elections. I no longer utter "glitch" or "programming error". I seek to describe specifically what happened in clear and understandable terms, and I do not accept responsibility-free excuses, -- er I mean explanations.
Along this line, I have recently witnessed such egregious practices under the word "audit", including a head moderator working as an official auditing his/her own head moderator's return, that I now place the word in quotes. We really, really need truly thoroughgoing examinations of election administration, ballot verifiability, machine function (no more lifeguards at the bottom of Niagara Falls - look thoroughly at machine function and security BEFORE the election, and then monitor ballots for voter intent AFTER the election?) -- there is a tremendous amount that serious and dedicated individuals can learn and apply to Six Sigma elections.
I am not attached to the Six Sigma continuous improvement model at all, nor do I work for anyone perpetuating it -- but 6 errors per million? that's music to my ears, that kind of commitment.
So -- if it's not an audit, let's call it a ballot examination (manual exam? )or something else, but preserve the word audit for something truly worthy of citizens' respect.
(Message edited by ctwatcher on March 10, 2008) |
Jean Braun
Voting Rights Forum Participant
Username: Jean_b
Post Number: 2
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 10, 2008 - 1:56 pm: |
|
I don't see the problem with ballot secrecy. If you don't have an unique ID for every ballot, Christine, how can you tell whether the record is a record of votes and not just a series of data? There need be no connection to voter registration; the voter enters the voting box with a random choice of voting boxes, BUT with the option of noting that ballot ID.
You may be right about the choice of the term "audit." At the same time, it is a phrase which is popularly understood to be a check on the integrity of a record. Perhaps there is room for some modification on the term here, to distinguish it from what you understand to be an audit. There is a great need for simplicity. |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4741
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 10, 2008 - 2:04 pm: |
|
Whatever is going on so far it is definitely not an audit or even a sincere attempt at an audit. There is too much improper use of this term.
I'm sorry to see some folks who espouse competent use of sampling statistics for partial recounts call suggest what they are proposing would be an "audit". |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 365
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 10, 2008 - 4:27 pm: |
|
How can you tell whether the ballot is a record of a vote and not a series of data?
What are we talking here - optical scan?
Ballot Chain of custody, ballot reconciliation vs. unused ballots and poll books and dual controls would be the "method" used now, I guess. Search secret ballot on this site for extensive discussion of the issues involved. |
Brant Lamb
Frequent Voting Rights Forum Participant
Username: Brantl
Post Number: 2005
Registered: 1-2005
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Tuesday, March 11, 2008 - 4:26 am: |
|
quote:I am not attached to the Six Sigma continuous improvement model at all, nor do I work for anyone perpetuating it -- but 6 errors per million? that's music to my ears, that kind of commitment.
The problem with this in regards to elections is that elections aren't like manufacturing, in manufacturing you know how things are supposed to turn out. And in manufacturing, for the most (overwhelming) part, everyone is on the same team.
How can you tell if there were or weren't 6 errors or less with an anonymous, untraceable, unverifiable-to-the-only-person-who-knows-what-it-should-be ballot system? This is your primary paradox, and dilemma. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 367
Registered: 12-2007
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Tuesday, March 11, 2008 - 5:07 am: |
|
Brant, I agree that you can't tell if there are 6 errors per million in a specific election result. So what else could be measured? 6 errors per million in a test run of ballots (we're nowhere close on that one).
What would you think of using such a standard for on timeliness of reports related to election verification, and auditing of e.g. ballot reconciliation by ballot clerk? Setting of standards for ballot chain of custody log, ballot storage situations, conditions of audit being met?
What I would NOT include would be "perfect election results where names on pollbook matched number of ballots -- what should be perfect is its timely reporting. 6 dead people per million registered voters/ Has possibilities.
The whole idea is to set the bar high and continue working to meet it, and at this point the cycle is ooops/excuse/oops/excuse. It's a different dynamic when the people know that they are being measured by a different yardstick. Where I agree it DOES get difficult is with temporary workers and volunteers.
If I accept your dilemma, I'm left with hand wringing or a nonsecret ballot. I'll do neither, A steep rate of improvement in any way accomplished, and a climate among ROVs and SOTS that does not accept excuses and sweetheart deals, is what I'd like to see.
(Message edited by ctwatcher on March 11, 2008) |
Jean Braun
Voting Rights Forum Participant
Username: Jean_b
Post Number: 3
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Tuesday, March 11, 2008 - 9:33 am: |
|
A ballot with an ID CAN be a secret ballot. I don't see what the argument is with that.
The dictionary defines "audit" first, as the professional examination of records, but secondarily, simply as the examination of records for verification, in which case a public counting of the election ballots at every precinct immediately after the polls close, can still qualify in the English language as a citizen audit of election ballots. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 370
Registered: 12-2007
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Tuesday, March 11, 2008 - 10:45 am: |
|
Jean, I would be most pleased if ballots were counted right after the voting, by any name. My purpose in examining the use of the word audit is that there is a singular lack of attention on the part of the voters to HOW their votes are counted. If the word audit is used for something that is anything but, they hear the word and stop listening. There is not meaningful verification in some of these so-called audits.
To provide a real life examaple on the present use of the word audit, the state of CT has something the SOTS says is to "confirm that the machines properly functioned on election day." This exercise calls for examining the machine cast ballots, sorting them visually based on the speculation as to whether or not the machine could read them, and then counting them. The only ballots not counted are overvotes or votes that are for some reason totally indecipherable. Then you have all ballots that are completely colored ovals, ballots where you can determine voter intent but are anomalously marked (checks, X's, circles around candidate, etc.). You do separate categories and then total the ballots and compare them to the machine cast total for the precinct on election day. You want to see a total that is equal to or less than the the reported machine cast ballots on election day. There is no examination of the number of voters vs. machine cast ballots, e.g. (that would only be problematic if the number of voters were less than the machine cast ballots -- they could be more, because of hand cast ballots etc.) the "audit" is very narrowly circumscribed.
Machine performance is idiosyncratic. A particular machine model is spec'd to perform a certain way, but e.g. a machine that had a bunch of rainy day wet ballots fed into it may not scan as sensitively as one that has not. Thus, we cannot assume based on common sense about what we believe the machine would count, that it did count it. There are many ways in which a machine may malfunction and miscount, and limiting the "audit" to a speculative visual examination of ballots that is never tested against assumption by running the ballots through the machine, and that may yield erroneous data, means the whole basis for comparison is specious. But it is called an audit. HAVA pays for it. As an example of the fallacy of equating human visual perception to the machine's ability to read a ballot, a perfectly visibly legible blue ballpoint pen vote may be perfectly unreadable to a voting machine with an infrared light reader, and if it is not set up to reject blank ballots, will be accepted but not machine counted).
As long as the ballots are presented in the room with a seal on them (and in some cases, they have been prsented otherwise), that substitutes for a chain of custody log examination. The audit law stipulates that the SOTS shall immediately send any reports with discrepancies to the UCONN consultant who is supposed to examine the machines. In 31 cases in the last round of audits in 2007, no escalation of discrepancies was done by the SOTS.
The final report is a series of numbers, and the ballots are resealed. For example, photos of the anomalous ballots might help the machine examiner to understand a scanning pattern problem, or identify a repeated pattern of "cue ballots" that some people say might be used to initiate code that runs the election differently. We require no documentation other than matching the ballots.
If the ballots match, there is no escalation. However, if the ballots have not been stored properly and they match because they have been tampered with, as long as the seals match, the audit proceeds as if nothing has happened. To date, ballot total matching equals ballot authenticity, even if there are seal problems (and the numbered tag type "zip seals" we use can easily be tampered with, I have read). The machine is being elevated to the source of indisputable information, which -- from looking at NH's machine votes discrepancies-- is not a reliable assumption.
The first step of the audit is to check the seal. If the seal is broken or missing and the audit proceeds, it is auditing ballots with no proper chain of custody, and tending to put a gloss of legitimacy on something that is not legitimate. Further, if the SOTS states that "we have the strictest audit law in the nation" but then does not escalate discrepancies and discover their source, what good is it?
(Message edited by ctwatcher on March 11, 2008)
(Message edited by ctwatcher on March 11, 2008) |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 371
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Tuesday, March 11, 2008 - 4:56 pm: |
|
To answer my own question about an audit (What good is it if escalation is not done on discrepancies?), there is still some residual good for reasons not documented necessarily in the audit itself: The exercise allows citizens to examine the ballots, and to observe the skills of the election officials responsible for conducting elections. It allows citizens to observe how these officials respond to discrepancies - Do they shrug them off? Do they do extra work so as to fully attend to them and resolve them? What crosses their radar as an issue, or not an issue?
So undeniably, some audit exercise tends to be better than no exercise, but if we are looking at how to maximize the return to citizens for their tax dollar, this is not perhaps a fully optimized way to spend the money. In CT, the GAE committee is hearing testimony and considering making the law stronger this year.
I recently observed two audits in CT, and two out of two of them involved sealed ballots and sealed machines stored in the same storage area, and rather than having access requiring a D and an R official, workers had access to the storage area.
The audits may be too much like lifeguards at the bottom of Niagara Falls -- if it only takes 5 minutes to compromise a machine or slip a seal and put on a counterfeit replacement, and routinely the room is accessed for other election related tasks, have we properly done the low tech protection of our election documents and equipment necessary to make the high tech part of the election security function even worth carrying out? |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4744
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Wednesday, March 12, 2008 - 2:14 am: |
|
quote:have we properly done the low tech protection of our election documents and equipment necessary to make the high tech part of the election security function even worth carrying out?
This is a crucial point. |
Brant Lamb
Frequent Voting Rights Forum Participant
Username: Brantl
Post Number: 2007
Registered: 1-2005
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Wednesday, March 12, 2008 - 4:19 am: |
|
With elections as they are currently stuctured (untraceable/unprovable-by-the-voter-ballots) you have a situation where an election can be sometimes detected as having been jiggered, but there is no way to correct it.
Let's try some hypotheticals:
Ballots are found with "serial numbers" (for want of a better term) that are not in the acceptable range of numbers that were used in that precinct. What do you do about them? They could have been handed out to unsuspecting voters with the express intent of geting them disallowed, couldn't they? They could be an accidental mix-up. Or, they could have been inserted into the boxes by someone without it being any kind of legitimate vote. (Who's to make sure you didn't mill around a precinct and insert a second ballot into the ballot box?)
You see a group of ballots in the box where they seem to have been dropped into the box as a dump, sequentially numbered or nearly so, in a big group, 50-70 ballots say and all pretty much voted (or exactly voted) the same, this doesn't pass the smell test, because you have a mix of voting opinion in your precinct and this has as much chance of happening as you have the chance of being struck by lightning 3 days in a row.
In your gut, you know it's wrong, but what can legally be done? I submit, probably nothing timely, if anything can be done at all.
Having said this, I think if someone can gain access enough that they can spoil 2 sequential runs of the same election I think that they're vitually guaranteed of getting the results that they want within those 2. |
Jean Braun
Voting Rights Forum Participant
Username: Jean_b
Post Number: 4
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, March 13, 2008 - 9:36 am: |
|
I can understand, now, Christine, why you have so much objection to arbitrary use of the term, "Audit". (Did you know that the two opponents of Senator Lieberman, in each of his two elections, received exactly the same number of votes?)
Also, I can see why Brant is focused on all the "cute" tricks involved in the logistical handling of ballots.
Elections involve control of millions - billions -
of tax dollars. I guess we should expect anything! And, fundamentally, we should question the control of the election process being in the hands of those who have vested interest in the outcome.
As so many of us are realizing, the problem is not knowing about it; the problem is what we can do about it.
The Voting Rights Act is extremely important here, as it criminalizes any official who denies an eligible voter the right to vote, or who fails to record and tabulate that vote.* Potential use of that law can be earth-shaking, but, as in Florida and Ohio that I know of, the courts can fail and even the House of Representatives can fail.
Mobilizing citizen action must be the key, and that is the real problem before us. A judge can ignore or set aside an individual Complaint; if an entire body of the community gets up in arms, almost any judge will pay attention to that.
This is why I believe so firmly in the old-fashioned community way of resolving things - having people gather at the neighborhood precinct on election night to count - as openly as possible - the official ballots for that precinct, and to have that count, once agreed upon by all present - be the official count for that precinct. The roll of the Board of Elections would be to receive and store the already counted ballots and to record the totals on the electronic machines.
Ideally, the machines will display each ballot on the BoE website, so that any voter can look up the machine's record of his/her vote and match it to the receipt (s)he received at time of voting.
My suggestion would be to recruit "counters" from the neighborhood junior high school, each counting the votes of one candidate and being potentially monitored by a representative of that candidate.
It would be a great civic lesson for the students, and assure a good representation of the public by parents accompanying their children.
In any case, all of us so passionately concerned about democratic elections need to decide on one secure way to guarantee election integrity, and to push as hard as we can for national legislation to guarantee the vote before November.
*Let me know if you want the legal citations from the Voting Rights Act. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 393
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, March 13, 2008 - 11:50 am: |
|
Jean, the one cute trick about audits that I did not mention is that Douglas Jones, a professor at University of Iowa, wrote about an apparently black ink pen made of I think red green and blue inks which was not readable with (an infrared??)certain types of scanners, but that to the eye read as black.
When people in elections tell you they've tested a lot of pens, after reading stuff like this, you kind of want to know the testing criteria!
I don't believe this would work on the visible light reader type scanners, but the take home message is that the what is visible to us may not be visible to machines -- and it's a good idea to be able to test to understand which are actually unreadable ballots to a particular machine, fed in a particular direction, marked in a particular way. While there are problems testing in test mode or presuming that one machine will read just like another, it would be a step better than we have now to be able to run ballots believed unreadable through and see if they were readable. With sealed machines it's probably not possible,but maybe as the second step in the whole go-round about ballot discrepancies in an audit.
I think there will be a great deal of overlap among the ballots from machine to machine that are read. There's a Doug Jones article here on this site (try the search feature) that painstakingly illustrates how an off-calibrated scanner may misread or not read at all a particular vote. Since I don't know of anyplace that programs machines NOT to accept undervotes, errors due to a missed or nonrecognizable vote for one candidate would not be detected unless there was enough discrepancy to cause someone to look into it and examine the ballots.
(Message edited by ctwatcher on March 13, 2008) |
Jean Braun
Voting Rights Forum Participant
Username: Jean_b
Post Number: 5
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Friday, March 14, 2008 - 12:14 pm: |
|
This is another reason not to accept machine records or results as official election results. You have very strong reasons to make that case.
Machines can be useful as repositories of votes and vote counts, and are probably here to stay, but it is very clear they cannot be relied upon for the integrity of an election.
Public interest in an election is always high; having many people gather at their neighborhood precinct to observe election counting can be very good fraud prevention. Recruiting teams of students to count sections of ballots (passing them from team to team)can enable the hand counting to be completed within a few hours at any precinct.
I don't see how, without this public observation, election counting can be accomplished without the probability of fraud. |
Brant Lamb
Frequent Voting Rights Forum Participant
Username: Brantl
Post Number: 2011
Registered: 1-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 17, 2008 - 3:55 am: |
|
Marjorie, if you want to discuss your method, please do it in a separate thread of your own. You keep sidetracking threads with this.
Making your own thread isn't difficult, just go to the bottom of the existing thread list and go to 'start new thread'. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 436
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 17, 2008 - 12:23 pm: |
|
I can confirm that it's off topic. If you look at the standards, Marjorie, they have to do with how you audit a professional IT system, and are presented for comparison to how voting systems are audited -- and machine function may be the focus, not necessarily election results per se.
To conduct a reputable audit, you must have data sources that are unimpeachable (or at least highly reliable in all the ways that materially affect the outcome/results).
If I had to pick a topic where your software idea MIGHT be road tested, I would suggest looking at groups that are attempting to do parallel voting exercises, which is something like exit polling. they are in a better position to say whether there could be a way to use your idea for their project goals.
Hope that helps. |
Brant Lamb
Frequent Voting Rights Forum Participant
Username: Brantl
Post Number: 2015
Registered: 1-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Wednesday, March 19, 2008 - 3:50 am: |
|
quote:Marjorie, if you want to discuss your method, please do it in a separate thread of your own. You keep sidetracking threads with this.
Making your own thread isn't difficult, just go to the bottom of the existing thread list and go to 'start new thread'.
This was true before and it's still true, if you want to continue to talk about this tangential topic, please start your own thread.
quote:It isn't that it has no connection, Marjorie, it just doesn't have much of a connection. I'm just asking you to start your own thread.
I don't believe that my tone was rude, or nasty. Blunt? Perhaps.
As for you having the solution, I think that's why you're having trouble here. People have pointed out problems with your 'the solution' and you've ignored them.
This is also still true.
You don't address the issues that people have brought up that there's no checking that whatever checking that you claim to be doing is being done each and every time, or that your software on the checking site could be compromised by the people that run your site.
Marjorie said (italics mine):
quote:What is wrong with you people here at Black Box Voting? Do you argue just to argue? Do you complain just to complain?
More time has been spent here arguing than just trying out the solution.
I will not be posting here anymore. I don't find the discussion to be of any value. May you all spin your wheels forever.
Apparently, we can't take you at your word. |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4788
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Wednesday, March 19, 2008 - 10:28 am: |
|
This thread is about auditing, and about what a real audit looks like. Whenever there appears to be an attempt to divert a thread away from its original focus (particularly by introducing inappropriate personal comments that might make someone disinclined to read the original material), then maybe the original topic and substance of that thread are particularly important.
I received email today about new "audit" recommendations. (They are not an audit, just a recheck of a certain percentage. This recount would be irrelevant or meaningless and misleading if there is no chain of custody of ballots.)
Marjorie, you are welcome here and please post about your system on a new thread. If you click on Forums on the upper left there is a special section there for different ideas about voting systems and technology. That would be a good place to start a new thread. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 450
Registered: 12-2007
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Wednesday, March 19, 2008 - 8:39 pm: |
|
Catherine Ansbro said:
I received email today about new "audit" recommendations. (They are not an audit, just a recheck of a certain percentage. This recount would be irrelevant or meaningless and misleading if there is no chain of custody of ballots.)
I think you are pointing to an incredibly important question about EAC-approved audits. As we have noted before, the term "audit" to an association of approved auditors means something entirely different from what some of the EAC-approved "audits" require, or how they are vetted.
One of the repeating themes of professional auditing association's documents and standards is an effort to reach an appropriate level of corroboration of the numbers presented. It's not enough for me to write on a piece of paper, "I have 3 trillion dollars in my checking account." You may completely believe me, but your job as an auditor is to be skeptical and seek corroboration. A real audit will spell out what is acceptable corroboration, or so it appears from reading these sites. You'd at least want to see copies of my checking account statements, and to go back and look at the deposits, their dates, and maybe even see some deposit slips or go to the check writers and ask them to verify the amounts. But my assertion of a $3 trillion balance you on principle would not buy.
Is it that way with our "post election audits?" Is anyone checking carefully??
In carefully done election audits, it's not enough for me, Suzy Moderator, to write on a piece of paper "Joe Schmoe got 3 trillion votes". On election night, someone should have to be a signed witness to that allegation, and maybe also witness a machine tape with 3 trillion votes for Joe Schmoe on it.
Now what happens to that piece of corroboration? Does the law require it to be saved? For how long? who must see it? Who is legally entrusted with storing it and under what conditions? And which pieces of paper are "escalated" to the next level of the election bureaucracy? IF this were a game of gossip, at what level does the independent corroboration -- the machine tape -- stop being a part of the explanation/documentation of what happened? At what level is it just numbers being whispered into the next ear, with no anchoring basis in observable fact?
When it comes to the audit, in what kind of audit is it enough for me to say "Trust me, Joe Schmoe got 3 trillion votes, see it says so on this piece of paper."
As a skeptical auditor, do I say the equivalent of, "May I see your passport, please?" Do I corroborate the underlying document? Does EAC require that I do so?
Does that machine tape saying 3 trillion votes still exist?? Under what conditions is it stored -- a desk drawer? A sealed envelope? Or do ancient statutes let it be destroyed because it slightly resembles an election document used long ago in the past, and no new language exists to protect it?
It is in sifting through such details that it starts to become clear whether we have based our allegations about what happened on election night on rock or on sand. Rhe election night exercise is something Bev has pointed out in detail in the past (I know this because I stumbled upon the information and it was so enlightening - maybe in the BBV book?).
Coming back to audits, I have to ask: what are EAC's standards on which an audit is evaluated? If we kick the tires on the audits that EAC has approved, how well do they stack up compared to a professional auditor's principles?
It isn't that each and every audit must be onerously detailed or involved. Audits can be utterly simple and elegant, I am reasonably sure.
But regardless of whether it is full of complex detail or a check with skeletal simplicity, it must withstand scrutiny.
If your state has an audit law, I do encourage you to kick the tires. It would not be good to discover you have a lemon. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 483
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 9:51 am: |
|
Bev, would you kindly move the discussion of votebankaccount to another thread?
Thread hijacking occurs easily in the excitement to discuss or present ideas (and in fact, you ought to pull some of my stuff from this weekend out of your ballot washing thread, imho, to keep your intent intact -- sorry).
I have just posted testimony re: CT audit laws in the CT thread, and would like to crosslink it here for comparison with Catherine's post about the headings used in the IT audit procedure guidelines.
I think the lengthy discussion of votebankaccount makes it difficult to follow on this particular thread's original thrust.
Thanks and hopefully Brant and Marjorie's discussion will not be interrupted, just moved. (Have fun, guys --just would like to move back to original topic, okay?) Maybe a post stating where to find the discussion would help anyone else following it, but not posting. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 484
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 11:04 am: |
|
Marjorie, I am sorry - no offense intended. I started the thread and would like to continue the discussion as begun - hence the request to Bev. Bev sometimes takes it on herself to move threads, and participants sometimes requests parts of threads to be moved. As you can see, I included my own inadvertent hijacking of Bev's thread in the comments, so I don't exclude myself from such a request either (a level of introspection I don't find common among dictators, but you may find me to be the exception)
That's the context for what I intended to be a polite and gracious request. I am sorry if it came across as dictatorial.
I think if, as background, you take a moment to read just the table of contents of my testimony here
http://www.bbvforums.org/forums/messages/73/73315.html?1206367396
and the first two pages or so, it might clarify why I would be making this request to keep the conversation intact.
I would appreciate it if you would just take a couple of minutes to do that and get back to me if you disagree.
The IT document was published in a document archives, not a general discussion thread, so I would think you might also get more takers for your discussion if it is taken out of the document archives.
Thanks.
(Message edited by ctwatcher on March 24, 2008)
(Message edited by ctwatcher on March 24, 2008) |
Bev Harris
Board Administrator
Username: Admin
Post Number: 7779
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 11:12 am: |
|
I moved Marjorie's comments to her thread in tech central, along with the responses to those.
Marjorie, please refrain from personal attacks. I will delete those posts. Limit discussion to issues, not personalities, and avoid hijacking threads. Chris Reid has been providing important work and insights, and I'd like to expand on those with on-point discussions rather than changing the subject. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 485
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 11:17 am: |
|
Bev, thanks. Please also consider moving comments starting from 3/17/08 3:55 Am Brant Lamb's response to MM through 3/19 3:28 pm C Ansbro. I think they may keep that thread's meaning intact. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 486
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 11:31 am: |
|
Coming full circle on the audit question, Catherine Ansbro made a terrific summary of an example of an IT audit upthread.
When exercises called audits are created by election chiefs in states, for a variety of reasons (related to elections/HAVA funding, probably most likely), it's very easy to glaze over and not look at the details. How well they work, though DOES depend on the details.
I live in CT, and we have a post-election audit law that has been said to be "perhaps the strictest audit law in the US". I decided to look at that statement further, and also to learn what exactly our audit law, and procedures, included.
What I did next was request a variety of documents so I could learn about the procedures for reporting on elections, the way the ballots were/are sealed and stored, and who gets what report. Then I looked at the audit procedures, and what information would be needed to evaluate the accurace of that report.
I took the questions I had, and asked in some cases the DSOTS to clarify whether my understanding was correct. I also read testimony or transcripts with information presented to our elections committee in the General Assembly, called the GAE.
I have learned so much about our elections, the law, the procedures, and how to find information placed in testimony. I also did something I had never done before: 1) went to a state legislative committee hearing and 2) submitted written testimony.
The documents are not everything I have learned or observed, but this is what I can post at this time. I would like to be able to post my post election audit report so people can see what one looks like and have a chance to raise their own questions about what I observed, but I have promised the citizens' group who conducted the audit to give them the report for their final report development.
I'm posting the testimony link here, because I think the questions raised about our audit, when compared to Catherine's IT audit summary here:
http://www.bbvforums.org/cgi-bin/forums/show.cgi?tpc=2197&post=42876#POST42876
raise some of what IMHO are important questions as to how well we are implementing audits, how arms' length they are, whether our laws sufficiently support the audits, whether the escalation (further investigation) of conditions uncovered in the audit is occurring at a professional enough level -- or whether our efforts are not sufficiently well done to serve the purpose for which they were intended.
Your observations are welcome - probably best in the other thread, if you want a convenient set of links to source documents (or here, if not)
http://www.bbvforums.org/cgi-bin/forums/show.cgi?tpc=73&post=44181#POST44181
but just wanted to call your attention to all this information as a whole.
{http://www.cga.ct.gov/2008/GAEdata/Tmy/2008HB-05888-R000312-Christine%20C.%20Reid-TMY.PDF,http://www.cga.ct.gov/2008/GAEdata/Tmy/2008HB-05888-R000312-Christine%20C.%20Reid-TMY.PDF}
|
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 489
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Monday, March 24, 2008 - 9:34 pm: |
|
To clarify, "conducted the audit" -- a citizens' group organized citizens to observe the state-conducted audits. My report was an audit observation report.
As it turned out, citizens had a lot of good observations about how those audits did and didn't work. More to come in next few weeks on that (maybe sooner - there's a vacation in there for the main author!) |
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 80
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Tuesday, March 25, 2008 - 2:09 pm: |
|
Here's a view, from a different angle, which I think deserves attention when thinking about, eg., chain of custody: from Bruce Schneier's blog on Security - 3/25/08 -
http://www.schneier.com/blog/
The Security Mindset
Uncle Milton Industries has been selling ant farms to children since 1956. Some years ago, I remember opening one up with a friend. There were no actual ants included in the box. Instead, there was a card that you filled in with your address, and the company would mail you some ants. My friend expressed surprise that you could get ants sent to you in the mail.
I replied: "What's really interesting is that these people will send a tube of live ants to anyone you tell them to."
Security requires a particular mindset. Security professionals -- at least the good ones -- see the world differently. They can't walk into a store without noticing how they might shoplift. They can't use a computer without wondering about the security vulnerabilities. They can't vote without trying to figure out how to vote twice. They just can't help it.
SmartWater is a liquid with a unique identifier linked to a particular owner. "The idea is for me to paint this stuff on my valuables as proof of ownership," I wrote, when I first learned about the idea: "Wouldn't it mean I could paint my liquid on your valuables, and then call the police..."
Really, we can't help it.
This kind of thinking is not natural for most people. It's not natural for engineers. Good engineering involves thinking about how things can be made to work; the security mindset involves thinking about how things can be made to fail. It involves thinking like an attacker, an adversary or a criminal. You don't have to exploit the vulnerabilities you find, but if you don't see the world that way, you'll never notice most security problems.
I've often speculated about how much of this is innate, and how much is teachable. In general, I think it's a particular way of looking at the world, and that it's far easier to teach someone domain expertise -- cryptography or software security or safecracking or document forgery -- than it is to teach someone a security mindset.
Which is why CSE 484, an undergraduate computer-security course taught this quarter at the University of Washington, is so interesting to watch. Professor Tadayoshi Kohno is trying to teach a security mindset.
You can see the results in the blog the students are keeping. They're encouraged to post security reviews about random things: smart pill boxes, Quiet Care Elder Care monitors, Apple's Time Capsule, GM's OnStar, traffic lights, safe deposit boxes, and dorm room security.
One recent one is about an automobile dealership. The poster described how she was able to retrieve her car after service just by giving the attendant her last name. Now any normal car owner would be happy about how easy it was to get her car back, but someone with a security mindset immediately thinks: "Can I really get a car just by knowing the last name of someone whose car is being serviced?"
The rest of the blog post speculates on how someone could steal a car by exploiting this security vulnerability, and whether it makes sense for the dealership to have this lax security. You can quibble with the analysis -- I'm curious about the liability that the dealership has, and whether their insurance would cover any losses -- but that's all domain expertise. The important point is to notice, and then question, the security in the first place.
The lack of a security mindset explains a lot of bad security out there: voting machines, electronic payment cards, medical devices, ID cards, internet protocols. The designers are so busy making these systems work that they don't stop to notice how they might fail or be made to fail, and then how those failures might be exploited. Teaching designers a security mindset will go a long way toward making future technological systems more secure.
===The rest is interesting too.
I think I'd qualify the remark about the engineer's perspective. In systems I worked on we were very much aware of and concerned about the environment - especially the active part that might be antagonistic...
Schneier, everyone reading BBV may - but surely should - know, is one of the planet's leading specialists in information security ... and a gifted author.
Abacus
|
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 495
Registered: 12-2007
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Tuesday, March 25, 2008 - 5:03 pm: |
|
Very interesting. I think for activists to use a security mindset allows them to identify problems with laws, procedures, and security around voting machines/elections.
I think that the concept "security mindset" has too much utility for obscuring other motives, because it is a "black box" definition. One can see that a person has/has not taken specific actions consistent with exploring security and taking care of flaws in security. As soon as we jump to mindset for an explanation, we provide plausible deniability "He just was too naive" rather than checking on the evidence of a person's action or inaction to see if they acted in full responsibility.
I think Scheier's ideas are very interesting, but am adding a caveat about getting stuff that is very airy-fairy and conceptual rolling around that gets in the way of what did they know, when did they know it, and what did they do about it or not. The rest is explanatory and we can leave that to the lawyers.
The document I posted above includes the description of ballots presented for an audit and is a very good example of actions taken, reasons given, and rules followed/not followed and by whom. The condition of the ballots was either 1) unknown by the SOTS -- why? or 2) acceptable to the SOTS and explained using gaps in the law, while simultaneously clearly in violation of the SOTS' own written procedures, which a reasonable person might presume were written for some reason other than to assuage citizens' groups who thought the audit law was nto stringent enough.
A discussion of security mindset in this case moves from the specific to the generic, and while fasciinating, risks obscuring exploration of the facts. It's a hypothesis that to me is beside the point of the facts for the moment. If I were fixing the problem, I could not fix someone's mindset but I could fix the locks on the vault.
Where in this case do you think the lack of security mindset would fit in? Legislature in defining the law? SOTS in response to report of cut ballot bag seals or failure to seal/examine the machines?
I think that while a security mindset is a useful tool for thinking through/analyzing the conditions that exist either in researching the facts around a contested election, or examining election administration practices that may lead to opportunities for fraud -- there is a risk of using the lack of security mindset for excusing other motives or misdirecting from what is frankly either negligent behavior, or deliberate manipulation of the law to provide cover for unethical behavior. I am confident that Schneier would be appalled at that use of his writing.
This is not a generic hypothetical statement on my part; this is a consideration of what real people are doing in real situations around our elections.
The security mindset is an instrument in the right hands, and a fig leaf in others.
Don't interpret my intensity of thought as coming at you, personally. I welcome your stepping to the side of the ideas and looking down at the Schneier ideas with me from the balcony. Great food for thought, Samuel. Thanks for posting it. |
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 81
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, March 27, 2008 - 7:24 pm: |
|
Christine,
Glad to dig in a bit. I think we have much the same overall view of election integrity; have much appreciated your work here, all along.
As to your take on “security mindset,” I’d like to offer some comments - not objections so much as explorations of some specifics.
In general I see you saying that “security mindset” is a useful thing in thinking about how the integrity of an election system can be considered. However, you see a possibility that misuse of the term can lead to rationalizations for breakdowns in integrity. Such rationalizations could lead to overlooking or covering up breaches of integrity.
This seems possible. Anything of the sort of course is indefensible. But this is not a criticism of “security mindset.” The possibility that a concept - or a term - may be misused as an excuse for a flaw in integrity is not a flaw in the concept.
Q [A quote from your text] ...think that the concept ‘security mindset’ has too much utility for obscuring other motives,.... One can see that a person has/has not taken specific actions consistent with exploring security and taking care of flaws in security. As soon as we jump to mindset for an explanation, we provide plausible deniability "He just was too naive" rather than checking on the evidence of a person's action or inaction to see if they acted in full responsibility. Q
Concur that any such conclusion would be an error. Indeed evidence of actions taken is what needs to drive a critique. Mental attitude has nothing to do with it. But it would be the “jump to mindset for an explanation” that would be the error, not the term itself.
If we are hiring - or firing - someone we have to make an assessment of the person’s skills and gifts. To label someone “...just naive...” isn’t productive. First, it’s just an opinion with no referents. Second, so what? Imho there’s not much in the way of remediation to be had except experience - and that doesn’t always work. But there is no SAT for Schneier’s ‘mindset.’ The course at the UW, you’ll note, is specific, working-level observation and logic, not abstractions with only woozy links to operational reality.
Pessimism is a mindset. But discussing a person’s pessimism quotient has nothing to do with whether he stuffed a ballot box.
Q The rest is explanatory and we can leave that to the lawyers.Q
Hoo! Can’t concur. You know this; haven’t Brant and Kurt braced us with it over and over? The legal world is dominated by words, by abstractions. Too often they deploy words, not in the interest of creating a good map of reality but in creating one that is slanted, preferably in ways not easily detected, in favor of some partisan conclusion. Dealing with this needs Mindset at full strength.
Q A discussion of security mindset in this case moves from the specific to the generic, and while fascinating, risks obscuring exploration of the facts.Q
You lost me here. Confusion about the level of abstraction at which a given term stands, and what its linkages to terms at higher and lower levels are, inevitably makes for misunderstanding, And, yes indeed, it makes exploring the facts harder.
Q It's a hypothesis that to me is beside the point of the facts for the moment. Q
Lost again. But concur that facts rule.
Q If I were fixing the problem, I could not fix someone's mindset but I could fix the locks on the vault.Q
Back on track again. Concur.
BUT BUT you might have to fire someone who is deficient in ‘mindset;’ else changing the locks isn’t necessarily going to fix your problem.
Q Where in this case do you think the lack of security mindset would fit in? Legislature in defining the law? SOTS in response to report of cut ballot bag seals or failure to seal/examine the machines? Q
Deficiencies in ‘mindset’ will of course crop up at all levels and in all corners of the field.
If you can establish that some of the players are deficient in ‘mindset’ you need to disconnect them from operations where their shortcomings might be dangerous. I think, often, your are stuck, You can’t easily unseat a legislator - eg, one with blind spots about DREs or with makers in his contributors’ list. Or wobbly election officials - the work that they should do, will not be done or will be done wrong. You will have to remove or isolate them as best you can, And you may just have to hope you’re lucky.
Strong audit designs would help.
Q I think that while a security mindset is a useful tool for thinking through/analyzing the conditions that exist either in researching the facts around a contested election, or examining election administration practices that may lead to opportunities for fraud. [Concur.] -- there is a risk of using the lack of security mindset for excusing other motives or misdirecting from what is frankly either negligent behavior, or deliberate manipulation of the law to provide cover for unethical behavior. Q
This is bedrock: Negligent behavior or covering such up is simply indefensible. Claiming deficient mindset is irrelevant.
But we can’t expunge the term from our vocabulary because someone might misuse it.
Q I’m confident that Schneier would be appalled at that use of his writing.” Q
No doubt; though, maybe, 'ruefully amused' would be closer. But such a use would be a gross misuse of his writing, not an objection to it.
Q This is not a generic hypothetical statement on my part; this is a consideration of what real people are doing in real situations around our elections. The security mindset is an instrument in the right hands, and a fig leaf in others. Q
I’m not so sure I want to know all about the right hands and the fig leaf...
but otherwise it seems to me we’ve done our job here? a security mindset is a label for a frame of mind, a mental approach to analysis of election systems, from marking pens to judicial, legislative and administrative pronouncements and including all processes and procedures concerned with voting integrity. Where breakdowns or flaws are discovered, it is logically inconsistent to excuse them as mindset failures; it is the specific operational-level actions and procedures that failed.
Q Don't interpret my intensity of thought as coming at you, personally. I welcome your stepping to the side of the ideas and looking down at the Schneier ideas with me from the balcony. Great food for thought, Samuel. Thanks for posting it. Q
Not in the least. It’s a pleasure to bounce ideas with you. Thanks for patience with long response.
Abacus
|
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 518
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, March 27, 2008 - 8:42 pm: |
|
Samuel,
The document I posted above includes the description of ballots presented for an audit and is a very good example of actions taken, reasons given, and rules followed/not followed and by whom. The condition of the ballots was either 1) unknown by the SOTS -- why? or 2) acceptable to the SOTS and explained using gaps in the law, while simultaneously clearly in violation of the SOTS' own written procedures, which a reasonable person might presume were written for some reason other than to assuage citizens' groups who thought the audit law was nto stringent enough.
A discussion of security mindset in this case moves from the specific to the generic, and while fasciinating, risks obscuring exploration of the facts. It's a hypothesis that to me is beside the point of the facts for the moment.
I rest my case! (grin)
Have a look at page 5 of the testimony posted in the link above -- this in a nutshell is the incident which became the source of the surrounding testimony recommending security changes in CT law. |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4824
Registered: 12-2004
Best of Black Box?
Votes: 1 (A keeper?) | | Posted on Friday, March 28, 2008 - 2:04 am: |
|
Samuel, reading Christine's testimony on page 4 and 5 will really be an eye-opener for you.
When you see what actually happened in CT maybe you will be more understanding of Christine's perspective about the danger of people using lame excuses who are paid to know better. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 520
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Friday, March 28, 2008 - 4:58 am: |
|
Thanks for reading it, Catherine.
Samuel, I NEED WANT APPRECIATE your careful thinking skills and attention. My own mind is so stretched thin right now from having spent the last few weeks pretty focused on trying to get something constructive going in my own state coming out of an audit I and I alone observed, that I'd rather persuade you to think about the issues I presented than let you persuade me to think about your careful read of my points, which are fully deserving of a reply.
Have a look and jump back on -- oh ye of security mindset! |
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 82
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Friday, March 28, 2008 - 4:14 pm: |
|
Catharine, Christine,
Sorry if this distracts from useful work. I don’t think there’s any real disagreement here.
Christine’s testimony is an impressive piece of work. One of its virtues is its relentless attachment to fact, it’s real avoidance of considerations of motivation of participants. This is the same territory I’ve staked out in many places, including in my last post - eg,:
Christine: “I think that while a security mindset is a useful tool for thinking through/analyzing the conditions that exist either in researching the facts around a contested election, or examining election administration practices that may lead to opportunities for fraud. [Sam: Concur.] -- there is a risk of using the lack of security mindset for excusing other motives or misdirecting from what is frankly either negligent behavior, or deliberate manipulation of the law to provide cover for unethical behavior.”
Sam: ”Negligent behavior or covering such up is simply indefensible. Claiming deficient mindset is irrelevant.” There’s no excuse for fraud. Good security design minimizes the chances of anything of the sort; good administration pursues it relentlessly...
But, Catharine, I can’t find any explicit reference to misdirection etc in Christine’s testimony, “...about the danger of people using lame excuses who are paid to know better.” Nonetheless, we are clearly in agreement about it being a real threat.
And, Christine, “I'd rather persuade you to think about the issues I presented than let you persuade me to think about your careful read of my points, which are fully deserving of a reply.”
I’m already on the same campaign you’re on, see no reason you need take time to go through the stuff I’ve posted.
Summing up: Have I missed something? I’m only suggesting that Schneier’s “security mindset” is a useful point of view when designing or critiquing voting integrity. Excusing malfeasance on the grounds that the perp’s “security mindset” is weak would of course be ridiculous. We don’t have to give up Schneier’s insights because someone might misuse them.
WRITTEN TESTIMONY -
Christine C. Reid
Wednesday, March 20, 2008 - GAE Committee
http://www.cga.ct.gov/2008/GAEdata/Tmy/2008HB-05888-R000312-Christine%20C.%20Rei d-TMY.PDF
Abacus
|
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 83
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Friday, March 28, 2008 - 4:21 pm: |
|
New Jersey Audit Bill
"...today at 1:06 AM, at the end of a grueling 12-hour lame-duck session, the New Jersey General Assembly made history by passing A2730/S507, the state's landmark post-election audit bill, by a bi-partisan vote of 53 to 19.
"The bill -- which is the first in the nation to require hand counts of enough votes to confirm the outcomes reported by electronic vote-counting systems independently of software -- now goes to Governor Corzine for his signature."
H. Stanislevic 1/8/08
http://e-voter.blogspot.com/2008/01/new-jerseys-post-election-bill-goes-to.html
Abacus
|
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 540
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Friday, March 28, 2008 - 10:44 pm: |
|
Sam, thank you for looking at the work. I truly appreciate it. Sounds like we are in agreement on the topic you raised.
I actually think a course like the one at UW would be fun -- and maybe a good thing for citizens serving in oversight roles. Being outside of something can help tremendously in analyzing it -- look at the discipline of doctors not treating relatives or friends, or anthropologists going to other countries to observe.
You key in on the aspect of the testimony that took a lot of rigor and time and discipline plus a number of days and nights of work -- the attachment to fact.
If I can rewind accurately, I'd say my resistance to the security mentality was a generic one -- it is a concept and concepts are sometimes conclusions and conclusions tend to be attached to arguments and arguments tend to be magnets for those facts that support them. or means of diverting poeple from the facts that don't (Catherine's reference to concepts applied in service of misdirection would fit here.)
I responded to Schneier's speculation as to whether it was innate or learnable, and in that vein, how it might be used to explain that someone did or did not have such a mindset. I wanted to focus not on debating such a conclusion or hypothesis, but on looking at what they actually did or did not do, free of concepts about why.
IMHO, if I am looking for the existence of the mentality, or nonexistence, I have glasses on looking for facts that fit/don't fit that concept.
IMPORTANT TO NOTE -- you were not coming from that particular direction. Sounds like that piece of Schneier's comments is not what you were keying in on.
So - I think there's tremendous utility in thinking, as Brant Lamb likes to call it, something like "the resident paranoiac".
And looking forward later to having a look at what NH codified. 100% of BS is still BS, so I'll be most interested in the actual audit procedures and law that go with the laudable percentages.
(Message edited by ctwatcher on March 28, 2008) |
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4835
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Saturday, March 29, 2008 - 2:28 am: |
|
By definition a "security mentality" would require being "the resident paranoiac".
Unfortunately such an attitude is actively discouraged by any comments with concern about observed facts/activities that "don't fit" being met with accusations that one is part of the "tin-foil" crowd--even if no accusations or claims are made.
You have to be able to imagine what someone with nefarious intent would do with the situations, systems and opportunities presented. |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 551
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Saturday, March 29, 2008 - 8:35 am: |
|
Once imagined and "fixed", the design becomes part of the election landscape to be exploited. Part of what a person engaged in malfeasance is doing is interacting with the design -- looking to see how to end-run it. So the design is both the end point of an inquiry (how to make things safer, based on imagining the existing system and potential changes to it/how they might be defeated) and the starting point of the next (how can I game this new system? What are its features and vulnerabilities?)
I'm interested in the possibility that more citizen oversight could make a bigger difference than dizzying levels and layers of security.
Hmm... what if our ballots were in a glass sided vault with lights on 24/7? That anyone could look in a window and see? not just locked - but locked and visible. Same with machines/cards. At time of audit, there should be no breached ballots or machines to deal with.
I guess a video cam in the vault 24/7 might do it, but I kind of like the idea of a public monument where the town's latest election's sealed ballots are on display in a secure way. Clear containers, windowed vault, and how about a video screen where you can scroll down and see who had acess to t he vault and when?
Couldn't resist -- I love the idea of an unpaid security force.
(Message edited by ctwatcher on March 29, 2008) |
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 84
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Saturday, March 29, 2008 - 12:10 pm: |
|
Excellent points, mesdames...
"Once imagined and "fixed", the design becomes part of the election landscape to be exploited. Part of what a person engaged in malfeasance is doing is interacting with the design -- looking to see how to end-run it. So the design is both the end point of an inquiry (how to make things safer, based on imagining the existing system and potential changes to it/how they might be defeated) and the starting point of the next (how can I game this new system? What are its features and vulnerabilities?)"
Yes; the ancient measure/counter-measure cycle. Another old truism: when the attacker is determined the defense always loses.
However, the defense can prevail for a time. So the strategy is to build it, maintain it, and revise it to keep abreast or ahead.
One component the defense can adopt: an active counter-attack capability. It should be designed to follow-up any sign of a probe and strike pre-emptively when the facts dictate. IT people use 'honeypots,' sites designed to tempt evildoers to visit or try to scam. This provides data for beefing up the defense or for appropriate active response.
So if a batch of marking pens that don't meet specifications turns up, the system doesn't just replace the pens; it actively looks into why/how this happens.
Abacus
|
Catherine Ansbro
Frequent Voting Rights Forum Participant
Username: Catherine_a
Post Number: 4845
Registered: 12-2004
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Saturday, March 29, 2008 - 2:22 pm: |
|
For the sake of adding other areas where this kind of thinking and expertise is common:
New Product Design tools, including--
Design for reliability techniques
Risk assessment techniques
There are a number of tools used by product or service designers in which teams try to come up with all the things that could possibly go wrong, assign a value for the likelihood of a given problem occurring, assign a degree of severity to the potential outcome for each thing that could go wrong, consider strategies (incl. costs) to weight possible ways of avoiding or managing the risk.
The tools that are used for this allow teams to quantify the risk and to make better-informed decisions about how to design a product, service or system.
One of the biggest problems we've got in election systems is that the people who order & use the systems (the legal customers--such as counties or election boards) are poorly informed as to both the range of risks and as to the likelihood of any given risk occurring. (For example, I'm sure most officials and voters in NH would have claimed that there was little or no chance of votes being changed by "ballot washing" in a way that could have an effect on an election. And they'd claim that there was no way an electronic voting machine could give inaccurate results, because they'd been "tested" and "certified" and the vendor says they're reliable, blah blah blah.)
They don't have a clue about many of the problems that are already occurring because no one is doing any meaningful system audits including such things as chain of custody. They are not even talking to their peers in other counties or states so awareness of problems is contained at the local level. (After all no one wants to admit to whatever problems they had--especially if their political bosses were controlling what happened.)
Everyone assumes that "nothing bad is happening here"--because no one looks in a meaningful way--then they pronounce that the risks are negligible or non-existent.
Add to that a continual effort to paint those who ask reasonable questions or who try to ascertain facts on the ground as some kind of fringe loonies or as unpatriotic. Or they claim that huge numbers of people would have to conspire in order for a given problem to occur.
This is no way to evaluate or manage risk associated with any new product or system. Can you imagine what would happen if cars were developed with this kind of attitude? |
christine c reid
Frequent Voting Rights Forum Participant
Username: Ctwatcher
Post Number: 553
Registered: 12-2007
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Sunday, March 30, 2008 - 2:00 am: |
|
However, the defense can prevail for a time. So the strategy is to build it, maintain it, and revise it to keep abreast or ahead
Theoretically, yes. Government agencies are driven by the statutes (in theory anyhow) that empower them to do particular activities. I'd be interested in hearing from people the best examples of government doing what you posit above, Samuel. I have the attitude that the way government works and its track record for responding in a proactive and continuously attentive way (via legislation) is generally not so hot, but the best thing to do is to look for the brightest examples of this approach working and learn from them, imho.
I have come to the conclusion that increasing the public's level of awareness and information about how their elections run (or don't), plus adding sunshine may be much cheaper than ever increasing technical sophistication. the low level of participation, knowledge of the system, awareness of how it might be gamed are constants in the definition of the problem, but you can't get a consulting contract by addressing them, so they remain....constants of the problem.
In our state, we created an audit law which the DSOTS says openly was done to "ease the transition" between lever machines and opscans.
In other words, it had a public relations/marketing value.
Most of the value from these audits has not been the audits themselves, because many of them were incorrectly completed (which didn't stop our SOTS from proclaiming that the machines had performed "perfectly", albeit with an imperfect and incomplete factual basis for making that conclusion).
It has comem not from HAVA money directly -- but from citizen volunteers who observed the audits and who were not operating under the election magician's oath and openly talked about what they observed in a report to the coalition that organized the observation/report system.
It has come from sunshine. The HAVA money without the free and volunteer citizens' participation would have been money authorized but not well spent, because the quality control and training for doing the audit was deficient. |
Samuel Scharff
Voting Rights Forum Participant
Username: Abacus
Post Number: 85
Registered: 8-2005
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Sunday, March 30, 2008 - 8:45 pm: |
|
Following Christine's line:
" have come to the conclusion that increasing the public's level of awareness and information about how their elections run (or don't), plus adding sunshine may be much cheaper than ever increasing technical sophistication. ..."
If the system was restricted to HCPB no one would have to explain any of the technical complexities and citizens would have a real chance of doing the job.......
Hee!
Abacus
|
Martha Bingham
Voting Rights Forum Participant
Username: Marieb
Post Number: 2
Registered: 3-2008
Best of Black Box? N/A
Votes: 0 (A keeper?) | | Posted on Thursday, April 3, 2008 - 1:29 pm: |
|
This is the URL for the Florida Secretary of State's Voting Section
http://election.dos.state.fl.us/votemeth/index.shtml
This is how Florida is apporaching state guidelines. |
|
|
