CSE603: Parallel and Distributed Processing

Course Information

Instructor

Dr. Jaroslaw Zola

Department of Computer Science and Engineering

Email: jzola@buffalo.edu Web: http://www.jzola.org/

For all email communication, please make sure to add prefix [PDP] to mail subject.

Course Description

This project-based course is designed to provide practical experience in building, debugging, testing and profiling end-to-end parallel applications. Throughout the semester, students work in small teams implementing either assigned or self-proposed projects focusing on a parallel architecture of their choice (e.g., GPGPU accelerators, shared-memory servers, distributed memory bare-metal or cloud-based clusters, etc.), and using their preferred programming model (e.g., CUDA, MPI, Spark, RPC-based APIs, etc.). The course puts equal emphasis on all aspects of the project execution, all the way from the conception and background research, through implement-test-benchmark loop, all the way to the end-product deployment and demonstration. The purely practical component of the course is complemented by milestone presentations and written reports by each team, and is culminated by the full-scale product presentation at the end of the semester.

Course Organization

The course is project based. You will form a team (and give it some cool name), and your team will work through the semester on building a parallel application. You may propose a project on which you would like to work or you may select a project from a set of projects provided by the instructor just before the beginning of the semester. Depending on your choice of the project, you will select the target parallel architecture for which you will be developing, and programming model and programming language you are going to use.

During the course, there will be three checkpoints during which you will be expected to submit written report as well as give presentation outlining your progress on the project. Specifically:

It is expected that you will be maintaining one document that will be evolving from checkpoint to checkpoint. At the same time, the bulk of your effort will be documented through commits to your GitHub/GitLab repository.

Course Prerequisites

Although the course currently has no specific prerequisites (that will change in the future), in order to succeed you must be already well familiar with basics of parallel processing. If you passed CSE 470/570 “Introduction to Parallel and Distributed Processing” you have the skills. If you have not taken CSE 470/570, make sure that you are well familiar with concepts like speedup, efficiency, scalability, you know how to test and assess them, and you know how to interact with HPC systems (bare metal or cloud, e.g., using SLURM). It goes without saying that since this is a project-based course the programming component will be critical to your success (you will have to write parallel code, choice of the programming model, e.g., OpenMP, TBB, MPI, Map/Reduce, Spark, CUDA, may be up to you). If you do not know what the acronyms in the previous sentence are, you probably should talk to your instructor to see whether this course is for you. If you are uncertain whether your background is sufficient to succeed in this course please contact your instructor ahead of time.

Program Outcomes

Upon completion of this course you should be able to understand and replicate a process that goes into building an efficient parallel processing application.

Course Requirements

The course has four requirements:

  1. Checkpoint 1 Students are expected to demonstrate a high level of proficiency with the tools, techniques and algorithms, and identify publications and potential initial source code that will form the basis of their project. The group is expected to submit and defend a roughly 5-page report surveying the space in which their project will be performed.
  2. Checkpoint 2 Students are expected to provide details of their approach and show the working code prototype. A roughly 5-page report should outline the group’s proposed design, any algorithms and techniques being introduced, as well as a strategy for evaluating the resulting project against the current state of the art. Also, any encountered obstacles should be clearly identified and the mitigation strategy should be discussed.
  3. Final Checkpoint Students are expected to provide a roughly 5 to 7-page report detailing their project, and summarizing all algorithms and techniques used. The report should provide detailed and extensive performance evaluation study using resources provided by CCR (addressing questions of scalability. e.g., weak/strong, and comparison against potential existing approaches).
  4. Code Base Students are expected to maintain solid code-base for the duration of the project. The code should maintain a high-quality standard in terms of portability and readability such that it can be demonstrated to potential employers. The code will be reviewed systematically by the instructor.
  5. Final Presentation Each group is expected to present their final project during the usual CSE Demo Day.

Grading Policy

The final grade will be assigned to the group using the weighted average: 15% report from Checkpoint 1, 15% report from Checkpoint 2, 40% report from the final Checkpoint, 30% quality of the final code. The final number-to-letter grade mapping will be done as indicated in the table below.

Score Grade Points
95-100 A 4.0
90-94 A- 3.67
80-89 B+ 3.33
70-79 B 3.0
60-69 B- 2.67
55-59 C+ 2.33
50-54 C 2.00
45-49 C- 1.67
40-44 D 1
0-39 F 0.0

In general, no incomplete grades (“IU” or “I”) will be given. However, in special circumstances that are truly beyond your control and justify incomplete grade, we will follow the university policy on incomplete grades, available here (for graduate students) and here (for undergraduate students).

Computing Resources

For the duration of the course you will be granted access to the resources (including storage) provided by the UB Center for Computational Research (CCR). CCR is the state-of-the-art HPC and data center hosting clusters, multi-core compute nodes, and compute nodes with GPGPU accelerators. It provides programming and execution environments supporting all types of parallelism covered in this course.

Academic Integrity

Academic integrity is critical to the learning process. It is your responsibility as a student to complete your work in an honest fashion, upholding the expectations your individual instructors have for you in this regard. The ultimate goal is to ensure that you learn the content in your courses in accordance with UB’s academic integrity principles, regardless of whether instruction is in-person or remote.

You must be familiar with the university and departmental policies on academic integrity! The university policies are available from https://www.buffalo.edu/academic-integrity/policies.html. The CSE policies are available from the CSE web page.

Any violation of these policies, including but not limited to cheating on any course deliverable (e.g., homework project, exam, etc.), will result in automatic failure of the course. There will be no leniency! If you decide to use a code from some external source, e.g., an open source project, you must include a proper and clearly visible attribution in your product (it is a good idea to contact your instructor to check if the code you plan to use is admissible).

Thank you for upholding your own personal integrity and ensuring UB’s tradition of academic excellence.

Accessibility Resources

If you have any disability which requires reasonable accommodations to enable you to participate in this course, please contact the Office of Accessibility Resources in 60 Capen Hall, 716-645-2608 and also the instructor of this course during the first week of class. The office will provide you with information and review appropriate arrangements for reasonable accommodations, which can be found on the web at: http://www.buffalo.edu/studentlife/who-we-are/departments/accessibility.html.

Counseling Service

As a student you may experience a range of issues that can cause barriers to learning or reduce your ability to participate in daily activities. These might include strained relationships, anxiety, high levels of stress, alcohol/drug problems, feeling down, health concerns, or unwanted sexual experiences. Counseling, Health Services and Health Promotion are here to help with these or other issues you may experience. You learn can more about these programs and services by contacting:

Counseling Services

Health Services

Office of Health Promotion

Sexual Violence

UB is committed to providing a safe learning environment free of all forms of discrimination and sexual harassment, including sexual assault, domestic and dating violence and stalking. If you have experienced gender-based violence (intimate partner violence, attempted or completed sexual assault, harassment, coercion, stalking, etc.), UB has resources to help. This includes academic accommodations, health and counseling services, housing accommodations, helping with legal protective orders, and assistance with reporting the incident to police or other UB officials if you so choose. Please contact UB’s Title IX Coordinator at 716-645-2266 for more information. For confidential assistance, you may also contact a Crisis Services Campus Advocate at 716-796-4399.

Please be aware UB faculty are mandated to report violence or harassment on the basis of sex or gender. This means that if you tell me about a situation, I will need to report it to the Office of Equity, Diversity and Inclusion. You will still have options about how the situation will be handled, including whether or not you wish to pursue a formal complaint. Please know that if you do not wish to have UB proceed with an investigation, your request will be honored unless UB’s failure to act does not adequately mitigate the risk of harm to you or other members of the university community. You also have the option of speaking with trained counselors who can maintain complete confidentiality. UB’s Options for Confidentially Disclosing Sexual Violence provides a full explanation of the resources available, as well as contact information. You may call UB’s Office of Equity, Diversity and Inclusion at 716-645-2266 for more information, and you have the option of calling that office anonymously if you would prefer not to disclose your identity.

Copyright 2021-2022 Jaroslaw Zola jzola@buffalo.edu