Cyberhelp

for Researchers & Teams

Connect to a Database Server

This Quick Start guide will walk you through establishing a connection to a database on SESYNC’s server. Access to your pursuit’s relational database management system (RDBMS) requires communication between a server application (PostgreSQL or MySQL) and a client application (RStudio, Jupyter, psql, etc.).

Jupyter Server

SESYNC provides access to remote JupyterLab sessions via a web browser. The Jupyter Project provides an environment for Python development, and SESYNC’s Jupyter Server adds direct connections to resources like shared file storage, databases, GitLab, and a compute cluster.

Share Synthesis Data Products

Choosing to put your data products in a long-term repository can:

Share Files and Folders with Anyone

SESYNC researchers and staff can provide a link for external collaborators to upload (donwload) files to (from) any research data directory they can access. We recommend this mechanism for receiving datasets from external collaborators: create a new folder and turn it into a public “file drop”, as described below.

RStudio Server

SESYNC provides access to a remote RStudio session, via a web browser, in order to work in R while directly connected to other SESYNC resources (file storage, databases, the cluster, etc).

Remote Meeting Participants

When you submit your travel planner to our travel office, please make sure to include a list of all remote participants that will be joining your meeting. You may add participants after the 8 week deadline, but we require that you notify us of all participants at least one week before the start of your meeting.

Run Python Scripts on the Cluster

The submission script lists the shell commands to be run from the cluster. To run Python code, the submission script can be as simple as:

Run R Scripts on the Cluster

You can use SESYNC’s RStudio Server to submit jobs to our cluster and view the resulting output. In the following example, you will learn how to:

Publish a Shiny App

To publish a R shiny application on the SESYNC server, your files will need to be copied from your working directory to the shiny-apps-data shared folder (/nfs/shiny-apps-data on RStudio Server). Please contact SESYNC IT staff if you would like to host an app on SESYNC’s Shiny Server.

Create Projects on GitLab

SESYNC offers private git hosting through our GitLab server at . When you connect to gitlab using your sesync name and password, you’ll see a dashboard of recent activity on projects that you are part of. If this is your first time connecting, it may be a little quiet.

The Compute Cluster

You can connect to the cluster through our ssh gateway service running at ssh.sesync.org or by submitting jobs through rstudio. The workflow for using a cluster is a little bit different from a typical run in R or python. In addition to your processing code, you must give the cluster a list of execution instructions and a description of the resources your analysis will require. Any output from your script will be written out to a file called slurm-[jobID].out and errors go to slurm-[jobID].err.

Bulk Uploads and Downloads by SFTP

You can upload and download data from your research data directory using an SFTP client. We recommend Cyberduck or WinSCP

Research Data Directory

SESYNC provides a large, shared file store to host data for all projects. Project participants have access to the research data directory for their project from our compute servers, a web portal, a desktop application for syncing, and SSH.

eBeam Whiteboards

SESYNC has installed the eBeam whitebaord capture software on all of our conference room PC’s and laptops.

Why isn't my research data directory in '/nfs'?

It is, or at least will be as soon as you need it! Any research data directory you have access to will be mounted to the filesystem at “/nfs” when you access it. If you have not touched any of the files in there for a while, it may have un-mounted and appear to be missing. So if you don’t see your “*-data” folder under “/nfs”, just navigate directly to the folder and it will instantly mount. For example, if your research data directory is “cooltrees-data”, then enter the full path as “/nfs/cooltrees-data” in the file browser or from the command line.

How do I connect to a video conference?

If you are a remote participant, see the below instructions the download the Skype for Business pugin and join your meeting.

Why does does my virtual machine show less memory than I requested?

SESYNC Windows client virtual machines are setup to use dynamic memory. What this means is that your virtual machine will show a different amount of memory available based on its current usage. You still have access to the full amount of memory allocated if needed. The virtual machine will grab more memory from the hypervisor when needed automatically.

Why does git show that all my files changed when I didn't change them?

Due to some quirks on our storage system your git repo may show that all of your files have modifications. If you perform a ‘git diff’ you will see a list that looks like:

What is a virtual machine?

A virtual machine is a Windows or Linux machine that runs on and shares computing resources with a physical machine known as a hypervisor. Virtual machines allow the deployment of multiple machines or services on one or several hypervisors to better utilize computing resources (CPU cores, memory, etc…)

What support does SESYNC provide for custom virtual machines?

SESYNC has the ability to deploy custom Windows and Linux Virtual Machines for use by groups. If there is a software or service needed that is not provided by our shared infrastructure, we can deploy a virtual machine to meet your needs.

Will SESYNC purchase software?

SESYNC can work with your group to obtain software that would be beneficial to your research while at SESYNC. All software purchased by SESYNC must be installed on SESYNC-owned equipment. We can provide virtual machines to use by your group members to access your purchased software for use.

When are the server maintenance windows?

When are the server maintenance windows?

What happens to my jobs during the maintenance window?

We’re sensitive to the fact that your jobs may need to run over our maintenance window and will take a reasonable effort to ensure they aren’t disrupted. In order to ensure as minimal disruption as possible, these are the steps that we take:

How do I manage my mailing list?

SESYNC uses the sympa mailing list server to manage mailing lists. You can manage your list by going to lists.sesync.org and entering your e-mail address and SESYNC password in the upper right.

Where are Shiny apps published?

Shiny applications hosted by SESYNC are publically available at the URL http://shiny.sesync.org/apps/<APPNAME>, where <APPNAME> is unique for each app. To get started on publishing an app, read the quick start guide.

How do I access Linux resources?

SESYNC linux resources are deployed on a private network at SESYNC and are accessed via our ssh gateway at ssh.sesync.org.

How do I schedule a conference call?

SESYNC offers the ability for your group to schedule conference calls outside of your onsite meetings. These calls can either be dial-in only, or audio-video. To schedule a call, please contact SESYNC it staff cyberhelp@sesync.org at least 7 days in advance of your call and we will configure a one time or standing meeting for you. Please let your participants know to keep a lookout for a meeting invitation from SESYNC’s IT staff. After you schedule your call, the following will happen:

How do I access my research data directory?

Navigate to https://files.sesync.org and log in with your SESYNC username and password. The folders listed under “External storages” are each a shared research data directory accessble to participants in the corresponding project.

When does SESYNC setup resources for my group?

For new groups, we generally follow the timeline below:

Should I use GitHub or SESYNC's GitLab?

If you already have projects on GitHub that you are working on, we prefer that you continue to use GitHub due to its open nature. We’ll gladly push and pull code from your public repository. We provide GitLab locally for projects that are just starting up, have sensitive data, or are not quite mature enough to be pushed out into the world.

What's the difference between git, GitHub, and GitLab?

The three are often a source of confusion.

Can code move between GitLab, GitHub, and Bitbucket?

Yes! You can push a local git repository to any new remote resource. Please note that only your source code will move. However, the additional features you use (e.g. wiki, issues, etc.) will need to be manually copied.

What is the compute cluster?

SESYNC’s computational cluster enables users to run medium-to-large scale analyses by distributing multiple, independent tasks across many computers. This setup is ideal for tasks that require applying the same algorithm or a parameter set over independent units in a large data set.

How do I access my Windows virtual machine?

SESYNC provides remote access to all desktop resources through a browser based Remote Desktop Protocol (RDP). Browse to https://desktop.sesync.org and login with your SESYNC username and password. Select one of the virtual machines to connect to its desktop (only machines you have permission to access are shown).

Do I have to use the cluster?

We highly recommend using the scheduled cluster for running all of your CPU-intensive or long running programs. Below is SESYNC policy for long running processes on our different types of resources:

How do I connect to a database from RStudio?

Access to a MySQL or PostgreSQL database from within R requires a database client and connection parameters (including a password) for the particular database you wish to access. This FAQ explains a recommended configuration that allows R to access a database on the SESYNC infrastructure. The instructions below are specifically for a PostgreSQL database, but analagous instructions apply to MySQL.

What's the procedure for having remote participants at my meeting?

When you submit your travel planner to our travel office, please make sure to include a list of all remote participants that will be joining your meeting. You may add participants after the 8 week deadline but we require that you notify us of all participants at least one week befor ethe start of the meeting.

What resources are available in the conference rooms?

SESYNC has five conference rooms and a large breakout space equipped with HD screens, projectors, whiteboards and an array of collaborative tools. Please note, depending on the size and requests of your group, your assigned meeting room may have a different table arrangement that waht is show in the pictures below.

How do I get a SESYNC account?

Anyone who is a PI on a science team, or is part of a long running pursuit, will receive an email with instructions on completing account setup prior to their first meeting.

How do I create an RStudio project with git?

RStudio projects are folders that contain project files and a special .Rproj file. To link an RStudio project with a git repository, follow these steps:

How do I change my SESYNC password?

Point your web browser to https://pwm.sesync.org.

What is my SESYNC username?

A SESYNC username is usually your first initial followed by your last name, (i.e. “John Smith” is jsmith). Common or very long names may not follow this pattern.

What resources are available?

SESYNC has an extensive set of computing resources and expertiese available for researchers. Download a high-level overview of all services and support SESYNC offers for general information, or scan the tables below for a quick reference. Direct all questions to cyberhelp@sesync.org.

How do I contact IT or research support staff?

Email cyberhelp@sesync.org with your question or support request.

Raster Change Detection Analysis with Two Images

Raster Change analysis with Two dates: Hurricane Rita

Making "dataspice" at #runconf18

As a perk of being an rOpenSci fellow, I recently got to attend the organization’s 5th ‘unconference’. This meeting brought together around 60 R users from around the world to spend a few days cooking up some new tools for the R community based on ideas discussed online leading up to the event.

Build a Shiny App to Browse MODIS Data

In preparation for our recent geospatial short course, I spent some time getting up to date on the new features in the leaflet R package. There are so many possibilities between the new add-ons in “base” leaflet, like inset mini maps and measuring tools, and even more functionality being added all the time in leaflet.extras, mapedit, and mapview.

Standardizing Non-standard Evaluation in R

Partway through her LTER Postdoc at SESYNC, ecologist Meghan Avolio ran into trouble manipulating her data on plant communities with dplyr functions. I had encouraged Meghan to modularize her scripts by writing functions for common steps in her pipeline (such as converting count data into rank-abundance curves). “You’ll love writing functions!” I said wrongly.

Writing Data Management Plans

Many funding agencies require proposals to include a section addressing plans for data management. This includes how you will handle data as it is being collected during the project, as well as plans for sharing and archiving once the project is complete. Here is a collection of resources we’ve found helpful for writing DMPs:

Images for Data Exploration in RShiny Apps

Photos, as a source of data, or to aid in the interpretation of data, can be a useful addition to RShiny applications. Here are two examples of using photo data: one that dispalys images from URLs, and another that uses species names to find pictures of animals.

RMarkdown

Extend your data pipeline with RMarkdown and Shiny.

Leaflet Maps

Make interactive maps in R using the leaflet package.

Basic SQL

Speak to a database in its native language.

Data APIs

Acquire data from websites and APIs.

Model Formulas

Write formulas for regression in R and Stan.

Plots in R

Craft publication-quality graphics with ggplot2.

Spatial NetLogo

Use spatial data in NetLogo ABMs.

Spatial R Packages

Manipulate geospatial data with open source tools.

Shiny Apps

Get interactive with the Shiny R package.

Text Mine

Carve your texts into structured data.

OSGeo

Meet the open source stack underlying geospatial data.

NetLogo Scripting

Implement open agent-based models.

Relational Databases

Make your data safe, scalable and relational.

Basic R

Start learning R in RStudio.

Basic git

Learn to use git with GitHub.

Basic Python

Start learning Python with Pandas and Scikit-learn.

Maps with R

Tour R packages that make static and interactive maps.

tidyr & dplyr

Get your data in shape with tidyr & dplyr.