Faculty Resources: Assessment Design & Tool Orientation

Audience: MSBAi faculty and course designers. Practical techniques for designing AI-aware assessments, building course materials with AI agents, and orienting students to the standard toolkit.

1. AI-Resistance Techniques for Written Assessments

Our primary synthesis-forcing mechanism is the oral defense (20-25% weight). But weekly assignments (30-40% of course grade) don’t have an oral component — they need their own AI-resistance. These techniques come from Manzoor (2026), whose open-laptop, AI-encouraged exams at Cornell have not been “one-shotted” by AI since 2023.

1.1 Exploit Sequence Model Weaknesses

LLMs generate what is plausible, not what is correct. Design questions that bait models into pattern-completing instead of reasoning:

1.2 Hand-Written Synthesis (In-Person Equivalent)

Manzoor requires hand-written submissions — students can use AI during the exam but must synthesize and write the answer by hand. This prevents copy-paste.

Our online equivalent: The oral defense serves this function. But for assignments without an oral component, consider:

• Require a short recorded explanation (2-3 min Loom/Zoom) alongside written work — students narrate their reasoning
• Process documentation requirements (AI Attribution Log) — make the thinking visible
• Sequential reveal problems — release data or constraints in stages so students can’t prompt the full problem at once

1.3 The “Attack Your Assessments” Exercise

Before finalizing any assessment, faculty should stress-test it with AI (see Assessment Strategy, Section 8):

1. Copy your assignment prompt into ChatGPT/Claude — can AI produce a passing answer?
2. If yes, identify where AI gets it right — those parts test recall, not reasoning
3. Redesign the vulnerable parts using the techniques above
4. Repeat each semester — AI capabilities change rapidly

New addition (from Manzoor): When testing, pay attention to whether AI gets the right answer for the wrong reason. Sequence models generate plausible explanations after deciding the answer — the explanation sounds convincing even when the answer is incorrect. Look for:

• Correct final answers with flawed intermediate steps
• Answers that would change if you swapped > for >= (but the model doesn’t notice)
• Reasoning that references domain knowledge not in the problem statement

Source: Manzoor (2026) — AI Innovation in Teaching Workshop

2. Building Course Materials with AI Coding Agents

Faculty don’t need to be web developers to build interactive course tools. Manzoor built haggleforme.computer (a full-stack negotiation simulation) entirely with AI coding agents — “I do not know modern JavaScript.”

2.1 The SPEC.md Workflow

1. Write a specification document (SPEC.md) describing what you want:
   • Product summary (what does it do?)
   • Scope (features to include/exclude)
   • Technical choices (suggest a stack or let the agent decide)
   • Example interactions (what does the student see?)
2. Give the spec to a coding agent:
   • Claude Code — terminal-based, reads the spec and builds the project
   • OpenAI Codex — similar capability, different model
   • v0.dev — UI-focused generation

3. Iterate on the result — describe what to change in natural language

4. Deploy — agents can also help with deployment (Vercel, Cloudflare Pages, etc.)

2.2 What Faculty Can Build This Way

2.3 Adapting Demos in Live Studios

Manzoor’s strongest teaching innovation: adapting demos in real-time during lectures using coding agents. For our Studio Sessions (90 min, weekly, live):

• Student-directed demo segment (10-15 min): Instructor takes a student’s question and uses a coding agent live to explore it — “Let’s see what happens if we change the loss function”
• Why it works: Students see the tool isn’t staged, the instructor is genuinely exploring, and surprising results emerge
• Risk management: If the agent fails, that’s pedagogically valuable too — students see AI limitations firsthand

Source: Manzoor (2026). GitHub repo with SPEC.md examples: github.com/emaadmanzoor/NBA6925-2026

3. Tool & Data Source Orientation for Faculty

All MSBAi courses share a standard toolkit. Faculty must be fluent in this stack and orient students during Week 1. Full details: program/tools.md.

3.1 Standard Toolkit Summary

3.2 Data Sources Available to Students

3.3 Faculty Week 1 Checklist

Before your first studio session, verify:

• You can open a .ipynb file in VS Code and run it on a Colab runtime
• Your GitHub Copilot is active and producing inline suggestions
• You can demo Copilot Chat (ask it to explain a code block)
• You know where WRDS data lives and how students access it (if your course uses it)
• Your assignment repos are on GitHub with clear README files
• You have an AI Attribution Log template linked in your syllabus (see Assessment Strategy, Appendix A)
• You’ve run the “Attack Your Assessments” exercise on at least your highest-weighted assignment

3.4 GitHub in MSBAi

Students create a GitHub account and set up VS Code before their first course (BADM 554, Week 0) and keep both for the whole program. The reasoning, the organization model, and the FERPA and Copilot analysis are in GitHub Strategy; student-facing setup steps are in program/tools.md. This section is the short version for faculty.

What the program owns

• The Business-Analytics-at-Gies GitHub organization, with two owners (Vishal Sachdev and Ron Guymon, named 2026-09-02). Faculty are added as members so they can reach shared templates and resources.
• Student onboarding: account creation, student verification for free Copilot, VS Code and extensions, all completed in BADM 554 Week 0.
• Two repository patterns students already know by Spring:
  • Team project repos are created by the instructor inside the organization from a template (badm554-team-template), private for the term, with mentors and instructor as collaborators. A departing teammate cannot delete graded work.
  • Portfolio repos belong to the student, on their own account, public when the student chooses. This is the showcase that accumulates across courses and the capstone.

What each course decides

• Whether to use GitHub at all. A course built on Colab, Power BI, or chat tools is fine. Students still have the account.
• Submission mechanics. BADM 554 uses a commit permalink for weekly work from Module 3: the Canvas submission is a link to one specific commit, so what is graded is exactly what was there at the deadline.
• Whether commit history is part of assessment. BADM 554 uses it at the Week 8 oral defense, where each student walks through what they built and when.
• Which AI tooling sits alongside it. Copilot is free and already set up; nothing requires a course to use it.

BADM 554 as the worked example (Fall 2026)

Module-by-module use is on the BADM 554 course page: individual repos in Weeks 1-5, provider-group shared repos in Week 6 for schema negotiation, the team repo as the final deliverable, and the oral defense over the commit history. If your course touches code or data, this is the pattern to copy or adapt; if it does not, the account still gives students a place to publish what they build with you.

Asks of faculty

• Send Vishal your GitHub handle so you can be added to the organization.
• If you plan to use team repos, tell your Lead Mentor at the pre-launch meeting so provisioning happens before Week 1.
• If a different platform serves public-facing portfolios better for your field, say so; the program position is a default, not a mandate.

3.5 Shared faculty-resources repository (planned, 2026-09-02)

A private repository in the Business-Analytics-at-Gies GitHub organization is planned as the shared home for code-facing faculty material: developer setup (VS Code, GitHub, Copilot, Lumen, GCP, DataCamp access), reusable notebooks and datasets, and helper scripts. It links to, and does not restate, the instructional-design material T&L maintains in Box (course maps, activity rosters, Canvas conventions, templates) or the program facts on this site. Scope is being confirmed with Cheng Li before the repo is created; see the 2026-09-02 entry in the decision registry.

4. Key References

Created for MSBAi Faculty Development — Gies College of Business