Stern School of Business, New York University

NYU ITS upgrades to research computing – new big data cluster

NYU/ITS (Information Technology Services) have recently added two major new facilities.
1) They have retired one of their HPC clusters and replaced it with a much more powerful cluster, mercer. Mercer is a 394-node, 5812-core heterogeneous cluster based on Intel x86_64 architecture. It is partially owned by several research projects. but time is available to the rest of NYU. They recently added SAS to their (very large ) list of supported software.
2) They have replaced their aging big data cluster with a 44 node hadoop cluster (Dumbo) running the latest Cloudera Enterprise software. The cluster has over 1400 terabytes of disk storage and 1170 processing cores. Initial benchmarks indicate it is very fast. It will be a joint use machine, used for both teaching in the Center for Data Science, and as a research facility. We are currently working with the HPC group to be able to move data very quickly (10gb link) from our storage into the big data file system. We are also working with them to enable SAS users to use the big data cluster as a high speed back-end for SAS running on their HPC cluster.
You can find more information about the NYU Research Computing Facilities here.