.. _spike/faq: FAQs ==== This file is also available on `GitHub `_, where it will be updated along with the development version of the code (and may be more responsive to common issues). Since there are so many different codes in play, it make sense to have a centralized place to address common questions/concerns without necessitating digging through the documentation/issues. I will amend this as new considerations come up, and I've tried to project for more common questions here. For more detailed questions/concerns about any of ``spike``'s dependencies, please refer to the original documentation for that code specifically. Space-based Imaging ------------------- 1. **The image processing steps aren't working and my data appear corrupted. Did spike cause this?** Probably not. If the data appear corrupted prior to any of the processing steps (tweak or drizzle/resample), try redownloading your images. If you are using the `MAST `_ batch retrieval, consider using the browser download, which may be more reliable. 2. **Roman hasn't launched yet; how do I access data to play with spike.psf.roman?** There are a lot of different groups dedicated to simulating data for the Roman Space Telescope. `Troxel et al. (2023) `_ has some nice examples of single detector data and you can generate your own Roman imaging with `STIPS `_ (`STIPS Development Team 2024 `_). The pipeline is very similar to JWST's, with the image from each of the detectors stored in its own file, so ``spike.psf.roman`` is, at its core, a modified version of ``spike.psf.jwst``. 3. **Tweakreg keeps failing, noting that I'm out of memory. How should I proceed?** This is a common issue with the WCS alignment steps, which are memory intensive, particularly for a large number of input files. The first (and easiest) solution is to simply specify pretweaked = True and skip that step altogether if the images are already sufficiently well-aligned for your purposes. Alternatively, you can alter the tweakparams input to change settings that may mitigate the load on your computational resources. You can also tweak the images yourself (which gives you more control over which files are included at any given point) and then, again, specify pretweaked = True when you run ``spike``. 4. **Why am I seeing "ValueError: WCSNAME 'TWEAK' already present...", and how do I resolve it?** Tweaking an image is a "destructive" process insofar as it irrevocably changes the FITS file. ``Drizzlepac`` specifically avoids overwriting this header key when it's present. You can either rerun the tweak step in ``spike`` using a different key (fed in via the tweakparams argument) or you can simply set pretweaked = True to skip the tweak step altogether. 5. **What file types are accepted by spike and why?** ``spike`` works by generating PSFs for individual exposures and then running them through the same processing and combination steps employed by the *HST*, *JWST*, and Roman pipelines to process images. This creates a location-specific drizzled PSF that is a resampled and combined in the same way as the images are in the final drizzled/mosaiced product. Because ``spike`` relies on doing this image combination step, the input files should be calibrated, but not yet combined -- e.g., cal, flt, or flc files and not i2d, drz, or drc files. Similarly, because ``astrodrizzle`` does not work with waivered FITS files, ``spike`` only takes as input multi-extension FITS files. 6. **I'm seeing an error that says "ValueError: Undefined variable 'uref' in string..." Why can't I tweak or drizzle WFPC2 imaging?** The WFPC2 data use a prior convention/format for astrometry and WCS. You must update the WCS to be compatible with ``drizzlepac``. There are `instructions on preprocessing WFPC2 data `_ from STScI; however, in my experience, data recently downloaded from MAST should not have ``stwcs.updatewcs.updatewcs`` run, and instead the CRDS keywords just need to be added to your path. ``spike`` also handles this automatically when WFPC2 data are used. Some other notes on this: The c0m files seem to behave more reliably than flt files as inputs. Additionally, if you are getting an error about celestial coordinates, ``spike`` may be reading the WCS from the header of one of the PSF models. Even with ``clobber=True``, make sure that your input file directory is clear of artifacts from previous (especially aborted) runs for best performance. Tweaking WFPC2 imaging may impact how ``drizzlepac`` reads the data, so I recommend only tweaking if necessary. 7. **My WFC3 data is in a filter not included in STDPSF and there are no suitable stars in frame for an empirical PSF. What should I do?** As STScI recommends against using ``TinyTim`` PSF models with WFC3 imaging, you can follow their suggested approach and use the library of empirical PSFs distributed via MAST. To access, simply go to `mast.stsci.edu `_, then under "Select a Data Category" choose "WFC3 PSF", specifying "UVIS" or "IR" under the "Waveband" dropdown before running the query. From there, you can filter the returned PSFs to best match your data. To use those single-frame PSFs with ``spike``, download the empirical PSFs from MAST into a local directory, making sure the files are renamed according to the convention in the ``spike`` documentation. You can then run ``spike.psf.hst`` as normal with ``method = 'USER'`` and the "usermethod" argument set to a string representing the path from your working directory to the directory where the STScI WFC3 PSFs are downloaded. spike ----- 1. **Why am I getting 'NameResolveError: Unable to find coordinates for name ...' when I feed spike.psf an object name?** First, check your internet connection. Name queries require access to the internet. The next most likely reason you'd get this sort of error is that the object name is not resolvable by Simbad or NED. Providing coordinates will allow ``spike`` to proceed. If searching for that name generally works, I have found that sometimes (perhaps due to updates on the CDS side of things?) searches that have worked in the past will temporarily return an error. In that case, you can either wait a bit and try again with a name search or you can provide the coordinates yourself to run ``spike`` immediately. As of v1.2.4, ``spike`` will prompt for coordinates if an object name is not resolved rather than raising an error. (If you would like to turn off this behavior, you will have to hard code "prompt = False" in either ``spike.tools.objloc`` or in the instances of ``spike.tools.objloc`` in whatever function you're using. I may add a formal option for this in the future, so feel free to contact me if this is something that would benefit you.) 2. **I am encountering a problem with one of the pipeline modules (spike.jwstcal, spike.romancal, or spike.stcal). How much has changed from the official Space Telescope release?** Each of ``spike.jwstcal``, ``spike.romancal``, and ``spike.stcal`` are subtly reworked versions of the STScI releases. Since namespace packages are not enabled (and would not be workable with the many interlocking dependencies) for ``jwst`` and ``romancal``, the relevant portions of the JWST and Roman pipelines are installed directly with ``spike`` with pared down dependencies and install requirements. The code has been reorganized and reworked to have a more straightforward file structure and get rid of dependencies that are extraneous for the tweak and resample pipeline steps, so none of the scientific functionality has been changed. If your error seems to come from one of these modules, you can always start with the `jwst `_ or `romancal `_ documentation to better understand the problem. Most interaction with these pipeline commands is through other ``spike`` functions, though, so it will also serve to toggle ``verbose = True`` and confirm that all prior steps are proceeding as expected (confirming the outputs look as you expect may help, as well). If the problem actually occurs at an intermediate step and the pipeline error is due to missing inputs, you can fix this by resolving the concern with that intermediate step (see the `spike documentation `_ or the other FAQ sections). If you encounter a bug that ultimately traces back to ``spike.jwstcal``, ``spike.romancal``, or ``spike.stcal``, and you would like to open an issue, please tag your issue with the "pipeline" label. Note that ``spike.jwstcal`` and ``spike.romancal`` are dependent on ``crds``, which means that the two environment variables CRDS_PATH and CRDS_SERVER_URL must be set according to the instructions `here `_. As of v1.2.2, ``spike`` allows users to toggle whether the STScI codes are used (if they are available in the environment) or the ``spike`` version of the pipeline modules via the "usest" keyword in each of the ``spike.psf`` functions. There is also the option to install ``jwst`` and ``romancal`` from source as extras when installing ``spike`` (e.g., ``pip install spike-psf[stdep]``). The "usest" option has not been tested for all OSes or versions of Python/dependencies and bug reports/version-dependent behavior reports are very welcome. 3. **I'm encountering an AttributeError when I enable parallelization and define my own function to generate PSFs. How do I fix this?** As it turns out, ``multiprocessing`` specifically requires that functions be *imported*. It is possible that you will have to create a dummy script that houses your custom PSF generation function. There's a good example of this on `StackOverflow `_ and in the `spike tests `_. 4. **I'm working in Jupyter Lab/Notebook, and my environment variables are not loaded/not found, impacting $TINYTIM/$WEBBPSF_PATH/$CRDS_PATH. How do I access them from Jupyter?** There are several ways to make sure that your environment variables are accessible. The various options are discussed in this `StackOverflow thread `_. I recommend following the example laid out in the ``spike`` `example_notebooks README `_, which amends your kernel.json to consistently read your environment variables directly from your startup file. 5. **Why am I getting some version of 'OSError: 8388608 requested and 0 written'?** This error pops up when your computer is out of storage. The imaging files that ``spike`` uses are already quite large, and the various PSF generation and drizzling steps create additional files that can take up quite a bit of space. There is little to do for this while ``spike`` is running, but you can toggle ``finalonly = True`` from ``spike.psf.hst``, ``spike.psf.jwst``, and ``spike.psf.roman`` to only retain the drizzled/resampled PSFs after the last step. By default, though, ``spike`` saves all intermediate products. 6. **Can I still use spike with Python 3.10?** In short, yes, though this is not recommended. In response to the deprecation of Python 3.10 support in ``drizzlepac >= 3.10``, beginning in v1.2.0, ``spike`` was updated to require Python >= 3.11 by default. If you would like to use Python 3.10, earlier versions of ``spike`` will work. You may also wish to handle things on an ad hoc basis (at your own risk) and install ``drizzlepac<=3.9.1`` via `GitHub `_ with a more recent version of ``spike``. In that case, dependencies from `spike/docs/requirements_version.txt `_, which includes specified versions for all required packages in a working Python 3.10.16 environment (last tested with ``spike`` version 1.1.0), may be a useful guide. You may need to update your local ``spike`` setup.py in this case. 7. **I'm seeing errors related to pkg_resources or importlib. How do I fix these?** In version 82.0.0, ``setuptools`` fully deprecated ``pkg_resources`` without offering an immediate one-to-one alternative or any backward compatibility. Instead, ``importlib`` is supposed to be a replacement to most common workflows, but the relevant modules are not available for Python < 3.13. Many libraries are now scrambling to update accordingly -- some by installing another tool, ``importlib_resources``; some by moving directly to ``importlib``, though the primary functions are not available in all Python versions; and some by requesting that individual users downgrade their ``setuptools`` installations. ``spike`` does not require you to install additional dependencies to workaround a newer version of ``setuptools`` and remains compatible with ``pkg_resources``, so the easiest user-side solution is to downgrade to ``setuptools==81.0.0``. It also comes with an ``importlib`` solution, but for users who want to use the newest version of ``setuptools`` and are not yet using Python > 3.13, this will require updating Python versions. 8. **I want to feed drizzlepac the keyword argument build = False via drizzleparams, but spike forces build = True. What should I do?** ``spike`` ensures that ``drizzleparams['build'] = True`` in order to preserve the formatting of its output files. The ``drizzlepac`` build parameter just designates whether the output is a multi-extension FITS file, with the alternative being individual files that house each of the extensions. In most cases, users will genuinely want ``drizzleparams['build'] = True``, but in the case of mosaics where the multi-extension output will be prohibitively large, it is still possible to use ``spike`` with ``drizzleparams['build'] = False`` though it will require modification to the source code and you will lose the WCS-based cropping functionality. Please open an issue (or send me an email) if you require ``drizzleparams['build'] = False``. TinyTim ------- If you plan to use TinyTim PSFs (for HST), TinyTim must be downloaded and installed separately following the instructions `here `_. 1. **Why does tiny1 result in a path error ('Open_pupil_table : Could not open ~/tinytim-7.5/\*.pup')?** First, confirm that the .pup file actually exists in your tinytim-7.5 directory. Assuming it does, this error is likely due to the syntax you used to set your `$TINYTIM` path variable. For bash shells, the quick fix should be to swap out '\~/' with an absolute path like '/Users/username/'. 2. **Why am I getting an HTTPS error when my PSF generation method is 'TinyTim_Gillis'?** If you don't already have the `Gillis et al. (2020) `_ code downloaded, the first thing that ``spike.psfgen`` tries to do is download the relevant script. An HTTPS error may come about if BitBucket is down or if there's a problem with your connection. If you have the Gillis et al. script downloaded locally, simply copy it to your working directory and the issue should resolve. STDPSF ------ 1. **Why am I getting an HTTPS error when my PSF generation method is 'STDPSF'?** The STDPSFs are read into ``photutils`` as URLs as needed, so there could be an issue connecting to the Space Telescope website for `HST `_ or `JWST `_ STDPSFs. If you can connect to the site through the link above, there may have been a change in the filename or path as those directories are not static and are sometimes updated. In that case, please either open an `issue `_ or send me an `email `_ so that I can amend the code accordingly. 2. **No STDPSF grid exists for my chosen WFC3 filter. Can I still use a pre-computed empirical PSF?** Yes, you can. You can download pre-computed empirical WFC3 PSFs from `MAST `_ (under "Select a Collection" > "WFC3 PSF"). Once you download these locally, you can feed ``spike.psf.hst`` ``method = 'USER'`` where ``usermethod`` will be the path to the directory where you downloaded those PSFs. If there are point sources in your image, you always have the option to compute empirical PSFs via ``spike`` with ``PSFEx`` or the ``photutils`` effective PSF. PSFEx ----- If you plan to use PSFEx to generate empirical PSFs, both SExtractor and PSFEx must be downloaded and installed separately following the instructions `here `_ and `here `_ respectively. Some notes on PSFEx/SExtractor installation for Macs: You can use ``brew install automake, libtool`` to add the GNU functions that PSFEx requires for installation. If FFTW is not already installed on your machine, you will need to follow the instructions for both single- and double-point versions (see these `instructions `_). If ATLAS is not already installed, you can bypass that installation altogether by downloading and installing OpenBLAS. Even with a successful install, my experience was that PSFEx could not find the correct directories for OpenBLAS, so in your PSFEx directory, you will want to run some variation of ``./configure --enable-openblas --with-openblas-incdir=/opt/OpenBLAS/include --with-openblas-libdir=/opt/OpenBLAS/lib`` when configuring the Makefile for PSFEx. There is also a bug in the PSFEx installation code -- discussed nicely `here `_ -- within PSFEx/src/levmar/compiler.h, you will need to change "finite" to "isfinite" before attempting to run ``make``. 1. **My output single-image PSFs look funny/there's an issue with my SExtractor catalog. How do I fix this?** The first step if you aren't happy with your PSFEx output is to try adjusting the SExtractor and PSFEx parameters in their respective config files. ``spike.psfgen`` uses the default settings for each of these codes unless an overriding user input is specified. As a result, the star catalog and subsequent PSF generation are not fine-tuned for any specific use case beyond the parameters that were altered for high-resolution, space-based images. Within spike/configs, there are example configuration and parameter files for PSFEx and SExtractor. These can be used as guides and can be *copied* and directly modified. (I recommend against modifying any of the files in spike/configs themselves unless you are interested in making global changes.) 2. **Why am I getting an 'ZeroDivisionError: float division by zero' error?** PSFEx relies on being able to detect stars in your images. In my experience, in fields (and/or filters) with few stars, it is possible that ``PSFEx`` will still run, but return PSF_FWHM = 0 and PSF_SAMP = 0 in the header of the resultant .psf file, as it did not actually have what to fit. You should first check those keys to ascertain whether this is the issue. If it is, I recommend either changing the fit parameters in the ``SExtractor`` and ``PSFEx`` configuration files or adopting a different PSF generation method. STPSF/WebbPSF ------------- If you plan to use STPSF -- formerly WebbPSF -- PSFs (for JWST and Roman), the relevant data must be downloaded and included in your path following the instructions `here `_.